WO2023173530A1 - 一种卷积运算加速器及卷积运算方法 - Google Patents

一种卷积运算加速器及卷积运算方法 Download PDF

Info

Publication number
WO2023173530A1
WO2023173530A1 PCT/CN2022/087794 CN2022087794W WO2023173530A1 WO 2023173530 A1 WO2023173530 A1 WO 2023173530A1 CN 2022087794 W CN2022087794 W CN 2022087794W WO 2023173530 A1 WO2023173530 A1 WO 2023173530A1
Authority
WO
WIPO (PCT)
Prior art keywords
convolution kernel
array
size
unit
convolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/087794
Other languages
English (en)
French (fr)
Inventor
缪向水
付嘉炜
何毓辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to US18/266,610 priority Critical patent/US12293804B2/en
Publication of WO2023173530A1 publication Critical patent/WO2023173530A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11C—STATIC STORES
    • G11C5/00—Details of stores covered by group G11C11/00
    • G11C5/02—Disposition of storage elements, e.g. in the form of a matrix array
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11C—STATIC STORES
    • G11C5/00—Details of stores covered by group G11C11/00
    • G11C5/06—Arrangements for interconnecting storage elements electrically, e.g. by wiring
    • G11C5/063—Voltage and signal distribution in integrated semi-conductor memory access lines, e.g. word-line, bit-line, cross-over resistance, propagation delay
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the invention belongs to the field of microelectronic devices, and more specifically, relates to a convolution operation accelerator and a convolution operation method.
  • Convolutional neural network is a form of feedforward neural network, which usually includes convolutional layers, pooling layers and fully connected layers. Convolutional neural networks are usually used to process image information. After the image is input to the convolution layer, local features are extracted through convolution operations, and then the model size is compressed through the pooling layer to reduce the amount of calculation. Finally, the features are integrated through the fully connected layer. get the output.
  • the convolutional neural network has the characteristics of local connection and weight sharing. Its parameter size is much smaller than the multi-layer perceptron model, and it has achieved good results in the field of image recognition.
  • the convolutional neural network has fewer parameters, during the forward propagation process, the convolution operation in the convolution layer generates most of the calculations in the network.
  • the convolution operation process is: the convolution kernel is applied to the input data Swipe with a certain step size, and each slide performs a multiplication and addition operation between the convolution kernel and the corresponding input data. This is slow in calculation and takes up a lot of space in traditional computing units (such as CPU and GPU).
  • the two-dimensional input information needs to be stretched into one dimension and then input to the array; in addition, if you want to make full use of the array units, you must use For rolling input, the degree of parallelism of the input is very low and the time cost is huge; if multiple convolution kernels are copied in the crossbar array for large-scale parallel calculation, a large amount of area will be occupied, and most of the units will not be used. The effect of increasing leakage current on calculation results.
  • the present invention provides a convolution operation accelerator and a convolution operation method to solve the problem of stretching a two-dimensional input into a one-dimensional, parallel input when performing convolution calculations on a planar memory array.
  • the problem is low degree and low array unit utilization.
  • the present invention provides a convolution operation accelerator, including: a three-dimensional non-volatile memory array and a control module;
  • the three-dimensional non-volatile memory array includes: P word line electrode layers; a bit line electrode layer is placed between any two adjacent word line electrode layers; any adjacent word line electrode layer and bit line electrode layer A non-volatile memory cell array is placed between them, and the non-volatile memory cell array is vertically connected to the word line electrode layer and the bit line electrode layer;
  • the word line electrode layer includes a plurality of word line electrodes arranged in parallel; the word line electrodes in the P word line electrode layers together form a word line electrode array;
  • Each column of non-volatile memory cells in the non-volatile memory cell array is connected to the same word line in the word line electrode layer connected to the non-volatile memory cell array; in the non-volatile memory cell array The non-volatile memory cells on each diagonal line are connected to the same bit line in the bit line electrode array connected to the non-volatile memory cell array; where, the diagonal line is in the non-volatile memory cell array diagonal lines parallel to corresponding diagonals in the array of non-volatile memory cells;
  • the size of the two-dimensional input data is M ⁇ N;
  • the control module is used to split the convolution kernel into k convolution kernel units of size 2 ⁇ c by rows, where k is a positive integer. ;Select k different sub-array units of size (M-2(k-1)) ⁇ N in the word line electrode array, and split them with k convolution kernel units of size 2 ⁇ c according to the convolution kernel
  • the sequence corresponds one to one; each convolution kernel unit and its N-c copies are stored in the two-layer non-volatile memory unit array between all two adjacent word line electrode layers in the corresponding sub-array unit;
  • the control module is used to split the convolution kernel into k convolution kernel units of 2 ⁇ c size and one convolution kernel of 1 ⁇ c size by rows. unit; select k sub-array units with a size of (M-2(k-1)-1) ⁇ N and one sub-array unit with a size of (M-2k) ⁇ N in the word line electrode array; k The sub-array unit of size (M-2(k-1)-1) ⁇ N corresponds one-to-one with k convolution kernel units of size 2 ⁇ c according to the order of convolution kernel splitting; each 2 ⁇ c The convolution kernel unit of the size of (M- Each word line electrode layer in the 2k) ⁇ N sub-array unit stores a 1 ⁇ c size convolution kernel unit and the 1 ⁇ c size convolution in one of the non-volatile memory unit arrays connected to it.
  • N-c copies of the kernel unit all non-volatile storage units in the non-volatile storage unit array that do not store the convolution kernel unit are set to a high resistance state; rows 2i-1 to M- on the two-dimensional input data
  • the data of row 2(k-i)-1 is applied to the corresponding word line electrode in the i-th (M-2(k-1)-1) ⁇ N sub-array unit in the form of voltage according to the corresponding coordinate information.
  • i 1,2,...,k; apply the data from row 2k+1 to row M on the two-dimensional input data in the form of voltage according to the corresponding coordinate information to the above-mentioned size (M-2k) ⁇ N on the corresponding word line electrode in the sub-array unit;
  • the three-dimensional non-volatile memory array is used to implement the dot multiplication operation of the convolution kernel unit and different parts of the two-dimensional input data in parallel based on the non-volatile memory cell array, and output the convolution kernel unit and the two-dimensional input data in parallel through the corresponding bit line electrode layer.
  • the sum of the dot multiplication results of the corresponding parts of the two-dimensional input data is used to realize the convolution operation of the convolution kernel and the two-dimensional input data;
  • each row of convolution kernel data of the convolution kernel unit is sequentially stored on the corresponding diagonal line of the corresponding non-volatile storage unit array, and then moved horizontally along the convolution kernel sliding direction to the adjacent N-c diagonal lines again.
  • the row of convolution kernel data is stored, and each layer of non-volatile storage unit array stores the convolution kernel data of the corresponding row in the convolution kernel unit and N-c copies of the convolution kernel data.
  • the three-dimensional non-volatile memory array is used to calculate all adjacent sub-array units based on the above-mentioned size of (M-2(k-1)) ⁇ N.
  • the two-layer non-volatile memory cell array between the two word line electrode layers implements the dot multiplication operation of the 2 ⁇ c size convolution kernel unit and different parts of the two-dimensional input data in parallel, and then passes through the corresponding bit line electrode
  • the layer parallel outputs the sum of the dot multiplication results of the 2 ⁇ c size convolution kernel unit and the corresponding part of the two-dimensional input data;
  • the control module is also used to output the bit lines at the same relative position in each selected sub-array unit respectively. Summarize and obtain the convolution operation results of the convolution kernel and the two-dimensional input data;
  • the three-dimensional non-volatile memory array is used for all phases in the sub-array unit based on the above size of (M-2(k-1)-1) ⁇ N.
  • the two-layer non-volatile memory cell array between the two adjacent word line electrode layers implements the dot multiplication operation of the 2 ⁇ c size convolution kernel unit and different parts of the two-dimensional input data in parallel, and then passes the corresponding bit
  • the line electrode layer outputs the sum of the dot multiplication results of the convolution kernel unit of size 2 ⁇ c and the corresponding part of the two-dimensional input data in parallel; at the same time, based on the non-volatile sub-array unit of size (M-2k) ⁇ N
  • the linear memory unit array implements the dot multiplication operation of the 1 ⁇ c size convolution kernel unit and different parts of the two-dimensional input data in parallel, and then outputs the 1 ⁇ c size convolution kernel unit and the two-dimensional input data in parallel through the corresponding bit line electrode layer.
  • control module is also used to summarize the corresponding bit line outputs in each selected sub-array unit to obtain the convolution operation result of the convolution kernel and the two-dimensional input data.
  • the three-dimensional non-volatile memory array has a multi-layer stacked structure, which is formed by sequentially stacking a word line electrode layer, a non-volatile memory cell array and a bit line electrode layer; between each word line electrode layer, the word line electrode The bit line electrode layer and the bit line electrode layer are all parallel to each other.
  • control module includes: an input signal generating device, an output signal generating device and an output signal reading device;
  • the input signal generating device and the output signal generating device cooperate with each other to apply voltages on the word line electrodes and bit line electrodes corresponding to the sub-array units, select the corresponding non-volatile memory cells, and change the selected non-volatile memory cells.
  • the conductance value of the storage unit allows it to store the corresponding convolution kernel data
  • the input signal generating device is also used to apply the two-dimensional input data in the form of voltage to the corresponding word line electrode in the above-mentioned sub-array unit according to the coordinate information of the two-dimensional input data;
  • the output signal readout device is used to detect the output current on the bit line electrode and convert it into a voltage signal as a result of the convolution operation.
  • the read voltage of the input signal generating device is less than or equal to the current set voltage value on the non-volatile memory unit to ensure that the conductance value of the non-volatile memory unit is not changed during the convolution operation.
  • the above-mentioned convolution operation accelerator is used to implement the convolution operation in the convolutional neural network.
  • the above-mentioned control module is also used to adjust the conductance value of each non-volatile memory cell by applying a write voltage on the corresponding word line electrode and bit line electrode, so as to adjust the convolutional neural network.
  • the parameters of the convolutional neural network are updated, and the convolution operation result of the previous round of the convolution operation accelerator is used as the input of the next round of convolution operation of the convolution operation accelerator.
  • the absolute value of the write voltage difference applied on the corresponding word line electrode and bit line electrode is the corresponding set voltage value
  • the absolute value of the write voltage difference applied on the corresponding word line electrode and bit line electrode is the corresponding reset voltage value.
  • the present invention provides a convolution operation method based on the above-mentioned convolution operation accelerator, including the following steps:
  • the convolution kernel is split into k convolution kernel units of size 2 ⁇ c by row, and k is a positive integer; K different sub-array units of size (M-2(k-1)) ⁇ N are selected from the word line electrode array and combined with k convolution kernel units of size 2 ⁇ c according to the convolution kernel splitting order.
  • the convolution kernel is divided into k convolution kernel units of 2 ⁇ c size and one convolution kernel unit of 1 ⁇ c size by row; in the words Select k sub-array units with size (M-2(k-1)-1) ⁇ N and one sub-array unit with size (M-2k) ⁇ N in the line electrode array; k sub-array units with size (M -2(k-1)-1) ⁇ N sub-array units correspond to k 2 ⁇ c size convolution kernel units one-to-one according to the convolution kernel splitting order; each 2 ⁇ c size convolution unit
  • the core unit and its N-c copies are stored in the two-layer non-volatile memory cell array between all two adjacent word line electrode layers in the corresponding sub-array unit; for the above size, it is (M-2k) ⁇ N
  • Each word line electrode layer in the sub-array unit stores a 1 ⁇ c size convolution kernel unit and N-c of the 1 ⁇ c size convolution kernel unit in one of the non-vola
  • the dot multiplication operation of the convolution kernel unit and different parts of the two-dimensional input data is implemented in parallel, and the corresponding parts of the convolution kernel unit and the two-dimensional input data are output in parallel through the corresponding bit line electrode layer.
  • the sum of the results of the dot multiplication operation is used to realize the convolution operation of the convolution kernel and the two-dimensional input data;
  • the size of the two-dimensional input data is M ⁇ N; after each row of convolution kernel data of the convolution kernel unit is sequentially stored on the corresponding diagonal line of the corresponding non-volatile storage unit array, it is horizontally moved along the convolution kernel sliding direction. Move to the adjacent N-c diagonal lines and store the row of convolution kernel data again, realizing that each layer of non-volatile storage unit array stores the convolution kernel data of the corresponding row in the convolution kernel unit and the convolution kernel. N-c copies of data.
  • step S2 includes: based on all two adjacent word line electrodes in the sub-array unit with the size of (M-2(k-1)) ⁇ N
  • the two-layer non-volatile memory cell array between the layers implements the dot multiplication operation of the 2 ⁇ c-sized convolution kernel unit and different parts of the two-dimensional input data in parallel, and then outputs 2 ⁇ in parallel through the corresponding bit line electrode layer.
  • the sum of the dot multiplication results of the c-sized convolution kernel unit and the corresponding part of the two-dimensional input data is summarized respectively, and the bit line outputs at the same relative position in each selected sub-array unit are summarized to obtain the convolution kernel and the two-dimensional input data.
  • the result of the convolution operation of the input data
  • step S2 includes: based on all two adjacent words in the sub-array unit of size (M-2(k-1)-1) ⁇ N
  • the two-layer non-volatile memory cell array between the line electrode layers implements the dot multiplication operation of the 2 ⁇ c size convolution kernel unit and different parts of the two-dimensional input data in parallel, and then outputs it in parallel through the corresponding bit line electrode layer.
  • the sum of the dot multiplication results of the convolution kernel unit of size 2 ⁇ c and the corresponding part of the two-dimensional input data at the same time, based on the non-volatile storage unit array in the sub-array unit of size (M-2k) ⁇ N, After implementing the dot multiplication operation of the 1 ⁇ c size convolution kernel unit and different parts of the two-dimensional input data in parallel, the 1 ⁇ c size convolution kernel unit and the corresponding parts of the two-dimensional input data are output in parallel through the corresponding bit line electrode layer.
  • the sum of the dot product operation results; and the corresponding bit line outputs in each selected sub-array unit are summarized to obtain the convolution operation result of the convolution kernel and the two-dimensional input data.
  • the present invention provides a convolution arithmetic accelerator, in which each word line electrode connects one column of the matrix-arranged non-volatile memory cells, and each bit line electrode tilts the matrix-arranged non-volatile memory cells.
  • the storage units are connected together; the input data of each word line can be multiplied and accumulated with the upper and lower convolution kernel units to realize the natural sliding of the convolution kernel unit in the y direction in the two-dimensional input; the inclined bit lines and convolution Multiple copies of the kernel in the non-volatile storage array of each layer can multiply an input data with the convolution kernel data at different positions in the same convolution kernel unit, thereby realizing the multiplication of the convolution kernel unit in the two-dimensional input x Natural sliding in the direction, so the convolution calculation can be completed in parallel at one time without expanding the two-dimensional input into one dimension.
  • the data reuse rate and the degree of parallelism of the operation are both high and the operation speed is fast; compared with the traditional planar array
  • the present invention utilizes the high integration characteristics of the three-dimensional array to accommodate more units in a smaller area, perform more calculations, and greatly improve the utilization of the array units.
  • the convolution operation accelerator provided by the present invention takes into account the odd and even conditions of the convolution kernel behavior for splitting and storage, and can realize the convolution operation of different split parts and corresponding two-dimensional data in parallel, making full use of It uses the computing resources of the three-dimensional non-volatile memory array and has faster operation speed.
  • the convolution operation accelerator provided by the present invention can be used to realize the function of the convolutional neural network, and map the input, weight and output of the neural network to the input voltage, conductance and output current of the three-dimensional non-volatile memory.
  • the physical calculation process at the array level can naturally match the convolution kernel sliding operation process that exists in a convolutional neural network, and can retain the position information of the two-dimensional data once Complete the convolution operation instantly and obtain two-dimensional output data, which highly restores the calculation process of the convolutional neural network in the software.
  • it uses as few input and output ports as possible to achieve highly parallel calculation of convolution, which greatly improves resource utilization and calculating speed.
  • the present invention is based on the convolution operation of the three-dimensional non-volatile memory array. Compared with the traditional von Neumann architecture computing units such as CPU, GPU, etc., there is no data transmission speed limit between the memory and the computing unit, and realizes Integrating storage and calculation, it can be used to achieve efficient large-scale parallel convolutional neural network calculations.
  • Figure 1 is a schematic structural diagram of a convolution operation accelerator provided by the first aspect of the present invention
  • Figure 2 is a schematic structural diagram of a convolution operation accelerator provided in Embodiment 1 of the present invention.
  • Figure 3 is a schematic structural diagram of a three-dimensional non-volatile memory array provided in Embodiment 1 of the present invention.
  • Figure 4 is a top cross-sectional view of the three-dimensional non-volatile memory provided in Embodiment 1 of the present invention.
  • Figure 5 is a flow chart of a convolution operation based on the three-dimensional non-volatile memory array described in Embodiment 1 provided in Embodiment 2 of the present invention
  • Figure 6 is a schematic diagram of the convolution kernel sliding in the x direction of the input data to perform multiplication and addition operations in the three-dimensional non-volatile memory array natural matching convolutional neural network provided by Embodiment 2 of the present invention;
  • Figure 7 is a schematic diagram of the convolution kernel sliding in the y direction of the input data to perform multiplication and addition operations in the three-dimensional non-volatile storage array natural matching convolutional neural network provided by Embodiment 2 of the present invention
  • FIG. 8 is a front view of a three-dimensional non-volatile memory array with eight word lines in the lateral direction provided by Embodiment 3 of the present invention.
  • FIG. 9 is a top cross-sectional view of a three-dimensional non-volatile memory array with eight word lines in the lateral direction provided by Embodiment 3 of the present invention.
  • the present invention provides a convolution operation accelerator, which includes: a three-dimensional non-volatile memory array and a control module;
  • the three-dimensional non-volatile memory array includes: P word line electrode layers; a bit line electrode layer is placed between any two adjacent word line electrode layers; any adjacent word line electrode layer and bit line A non-volatile memory cell array is placed between the electrode layers, and the non-volatile memory cell array is vertically connected to the word line electrode layer and the bit line electrode layer; specifically, the three-dimensional non-volatile memory array is multi-layered.
  • the layer stack structure is formed by sequentially stacking a word line electrode layer, a non-volatile memory cell array and a bit line electrode layer; the word line electrode layers, and the word line electrode layer and the bit line electrode layer are all parallel to each other;
  • the word line electrode layer includes a plurality of word line electrodes arranged in parallel; the word line electrodes in the P word line electrode layers together form a word line electrode array;
  • Each column of non-volatile memory cells in the non-volatile memory cell array is connected to the same word line in the word line electrode layer connected to the non-volatile memory cell array; in the non-volatile memory cell array The non-volatile memory cells on each diagonal line are connected to the same bit line in the bit line electrode array connected to the non-volatile memory cell array; where, the diagonal line is in the non-volatile memory cell array diagonal lines parallel to corresponding diagonals in the array of non-volatile memory cells;
  • the size of the two-dimensional input data is M ⁇ N;
  • the control module is used to split the convolution kernel into k convolution kernel units of size 2 ⁇ c by rows, where k is a positive integer. ;Select k different sub-array units of size (M-2(k-1)) ⁇ N in the word line electrode array, and split them with k convolution kernel units of size 2 ⁇ c according to the convolution kernel
  • the sequence corresponds one to one; each convolution kernel unit and its N-c copies are stored in the two-layer non-volatile memory unit array between all two adjacent word line electrode layers in the corresponding sub-array unit;
  • the control module is used to split the convolution kernel into k convolution kernel units of 2 ⁇ c size and one convolution kernel of 1 ⁇ c size by rows. unit; select k sub-array units with a size of (M-2(k-1)-1) ⁇ N and one sub-array unit with a size of (M-2k) ⁇ N in the word line electrode array; k The sub-array unit of size (M-2(k-1)-1) ⁇ N corresponds one-to-one with k convolution kernel units of size 2 ⁇ c according to the order of convolution kernel splitting; each 2 ⁇ c The convolution kernel unit of the size of (M- Each word line electrode layer in the 2k) ⁇ N sub-array unit stores a 1 ⁇ c size convolution kernel unit and the 1 ⁇ c size convolution in one of the non-volatile memory unit arrays connected to it.
  • N-c copies of the kernel unit all non-volatile storage units in the non-volatile storage unit array that do not store the convolution kernel unit are set to a high resistance state; rows 2i-1 to M- on the two-dimensional input data
  • the data of row 2(k-i)-1 is applied to the corresponding word line electrode in the i-th (M-2(k-1)-1) ⁇ N sub-array unit in the form of voltage according to the corresponding coordinate information.
  • i 1,2,...,k; apply the data from row 2k+1 to row M on the two-dimensional input data in the form of voltage according to the corresponding coordinate information to the above-mentioned size (M-2k) ⁇ N on the corresponding word line electrode in the sub-array unit;
  • the three-dimensional non-volatile memory array is used to implement the dot multiplication operation of the convolution kernel unit and different parts of the two-dimensional input data in parallel based on the non-volatile memory cell array, and output the convolution kernel unit and the two-dimensional input data in parallel through the corresponding bit line electrode layer.
  • the sum of the dot multiplication results of the corresponding parts of the two-dimensional input data is used to realize the convolution operation of the convolution kernel and the two-dimensional input data;
  • each row of convolution kernel data of the convolution kernel unit is sequentially stored on the corresponding diagonal line of the corresponding non-volatile storage unit array, and then moved horizontally along the convolution kernel sliding direction to the adjacent N-c diagonal lines again.
  • the row of convolution kernel data is stored, and each layer of non-volatile storage unit array stores the convolution kernel data of the corresponding row in the convolution kernel unit and N-c copies of the convolution kernel data.
  • the three-dimensional non-volatile memory array is used to calculate all adjacent two sub-array units based on the size of (M-2(k-1)) ⁇ N.
  • the two-layer non-volatile memory cell array between the word line electrode layers implements the dot multiplication operation of the 2 ⁇ c size convolution kernel unit and different parts of the two-dimensional input data in parallel, and then passes through the corresponding bit line electrode layer.
  • Parallel output is the sum of the dot multiplication results of the 2 ⁇ c size convolution kernel unit and the corresponding part of the two-dimensional input data;
  • the control module is also used to summarize the bit line outputs at the same relative position in each selected sub-array unit. , obtain the convolution operation result of the convolution kernel and the two-dimensional input data;
  • the three-dimensional non-volatile memory array is used for all phases in the sub-array unit based on the above size of (M-2(k-1)-1) ⁇ N.
  • the two-layer non-volatile memory cell array between the two adjacent word line electrode layers implements the dot multiplication operation of the 2 ⁇ c size convolution kernel unit and different parts of the two-dimensional input data in parallel, and then passes the corresponding bit
  • the line electrode layer outputs the sum of the dot multiplication results of the convolution kernel unit of size 2 ⁇ c and the corresponding part of the two-dimensional input data in parallel; at the same time, based on the non-volatile sub-array unit of size (M-2k) ⁇ N
  • the linear memory unit array implements the dot multiplication operation of the 1 ⁇ c size convolution kernel unit and different parts of the two-dimensional input data in parallel, and then outputs the 1 ⁇ c size convolution kernel unit and the two-dimensional input data in parallel through the corresponding bit line electrode layer.
  • control module is also used to summarize the corresponding bit line outputs in each selected sub-array unit to obtain the convolution operation result of the convolution kernel and the two-dimensional input data.
  • the output of each sub-array unit is (M-2k) ⁇ (N-c), and the final convolution operation result can be obtained after corresponding addition.
  • the module splits the convolution kernel into k convolution kernel units of size 2 ⁇ c, and treats the three-dimensional non-volatile storage array with k ⁇ N word lines in the lateral direction as k sub-units with N word lines.
  • the array unit maps a 2 ⁇ c size convolution kernel unit at the corresponding position of each sub-array unit.
  • the two-dimensional input data will also slide on k array units.
  • the first array unit word line maps the data from the first row to the M-2(k-1)th row
  • the second array unit word line maps the data from the 2+1th row to the M-2(k-2)th row
  • the i array unit word line maps the 2i-1th row to the M-2(k-i)(i ⁇ 1,2,3,...,k)th row of data.
  • the control module is also used to summarize the bit line outputs at the same relative position in the k sub-array units to obtain the final convolution operation result.
  • the mapping method of k convolution kernels of size 2 ⁇ c is similar to the method when the size of the convolution kernel is 2k ⁇ c.
  • the difference is that in the horizontal direction Select a three-dimensional non-volatile memory array with (k+1) ⁇ N word lines, and regard the three-dimensional non-volatile memory array with (k+1) ⁇ N word lines in the lateral direction as k+1 with N sub-array unit of word line, the i-th array unit word line maps the 2i-1th row to the M-2(k-i)-1(i ⁇ 1,2,3,...,k)th row of data; for the remaining A 1 ⁇ c size convolution kernel unit is mapped to the k+1th array unit.
  • the non-volatile memory units above (or below) are mapped to the k+1th array unit.
  • the 1 ⁇ c size convolution kernel unit and its N-c copies are stored in the volatile memory cell array, and all non-volatile memory cells below (or above) the bit line are set to a high resistance state (considered as open circuit);
  • M-2(k-1)-1 word line layers are selected vertically, and (k+1) ⁇ N word lines are selected horizontally; when storing on the upper non-volatile memory cell array
  • the convolution kernel unit of size 1 ⁇ c is used, the data from the 2k+1th row to the Mth row in the two-dimensional input data are mapped on the 1st layer to the M-2kth layer from top to bottom.
  • the control module is also used to summarize the bit line outputs at the same relative position in the k+1 sub-array unit to obtain the final convolution operation result.
  • the convolution operation accelerator in this embodiment is shown in Figure 2, including: a three-dimensional non-volatile memory array and a control module, wherein the control module includes: an input signal generating device, an output signal generating device and an output signal readout device;
  • Figure 3 shows a schematic structural diagram of a three-dimensional non-volatile memory array, including word line electrodes 1, bit line electrodes 2, and non-volatile memory cells 3.
  • the input signal generating device is connected to the word line electrode, and the output signal generating device and the output signal reading device are connected to the bit line electrode.
  • V11 ⁇ V44 are applied to the word line electrode in the form of voltage as input, and I11 ⁇ I33 are the output currents obtained after the input voltage is applied to the non-volatile memory array.
  • Figure 4 shows a top cross-sectional view of the three-dimensional non-volatile memory.
  • the top cross-sectional view shown on the left is a two-layer structure of a word line electrode and a non-volatile memory cell array, with the non-volatile memory cell array above the word line electrode.
  • Word lines 1 to 4 are electrodes arranged in parallel, and non-volatile memory cells are evenly distributed on each word line.
  • Non-volatile memory cells 411 to 441 are connected to word line 1, and non-volatile memory cells 412 ⁇ 442 is connected to word line 2, non-volatile memory cells 413-443 are connected to word line 3, and non-volatile memory cells 414-444 are connected to word line 4.
  • the top view cross-sectional view shown on the right adds a layer of bit line electrodes to the top view cross-section view on the left. From bottom to top are word line electrodes, non-volatile memory arrays and bit line electrodes.
  • the bit line electrodes are arranged in parallel, but at a certain angle with the direction of the word line electrodes.
  • the non-volatile storage unit 411 is connected to the bit line 1
  • the non-volatile storage units 421 and 412 are connected to the bit line 2
  • the non-volatile storage units 431, 422 and 413 are connected to the bit line 3.
  • Cells 441, 432, 423, 414 are connected to bit line 4.
  • each word line connects one column of the matrix-arranged non-volatile memory cells, and each bit line tilts the matrix-arranged non-volatile memory cells. units are connected.
  • the input data of each word line can be multiplied and added with the upper and lower convolution kernel units to realize the natural sliding of the convolution kernel units in the y direction in the two-dimensional input.
  • the multiple copies of the tilted bit lines and convolution kernels in each layer of non-volatile memory arrays can allow the multiplication of one input data and the convolution kernel data at different positions in the same convolution kernel to realize the convolution kernel in two
  • the convolution calculation can be completed in parallel at one time without expanding the two-dimensional input into one dimension, and the operation speed is extremely fast.
  • the input signal generating device and the output signal generating device cooperate with each other to apply voltages on the word line electrodes and bit line electrodes corresponding to the sub-array units, select the corresponding non-volatile memory cells, and change the selected memory cells.
  • the conductance value of the non-volatile storage unit allows it to store the corresponding convolution kernel data
  • the input signal generating device is also used to apply the two-dimensional input data in the form of voltage to the corresponding word line electrode in the above-mentioned sub-array unit according to the coordinate information of the two-dimensional input data;
  • the output signal readout device is used to detect the output current on the bit line electrode and convert it into a voltage signal as a result of the convolution operation.
  • the input signal generating device is used to apply the voltage corresponding to the input data on the corresponding word line electrode, and at this time, the voltage amplitude does not exceed the set voltage of the non-volatile memory cell.
  • the output signal reading device is used to convert the output current on the bit line electrode into a voltage and read out the voltage value as a result of the convolution operation.
  • the above-mentioned convolution operation accelerator is used to implement convolution operations in convolutional neural networks.
  • the word line electrodes of the three-dimensional non-volatile memory array serve as the input of the convolutional neural network.
  • Each layer of word line electrodes corresponds to a row in the two-dimensional input of the convolutional neural network, and each layer of the non-volatile memory array corresponds to the convolution kernel.
  • a row in each layer of bit line electrodes corresponds to a row in the two-dimensional output of the neural network.
  • the input signal generating device is used to apply voltage on the word line electrode.
  • the applied voltage has two purposes. One is to use the voltage to represent the input data.
  • the current output is obtained to complete the forward propagation operation of the neural network.
  • the output signal generating device is used to apply voltage on the bit line electrode, cooperate with the word line electrode to select a non-volatile memory cell, and change the conductance value of the selected non-volatile memory cell.
  • the output signal readout device is used to detect the output current on the bit line electrode and convert it into a voltage signal as a result of the convolution operation.
  • control module is also used to adjust the conductance value of each non-volatile memory unit by applying a write voltage on the corresponding word line electrode and bit line electrode, so as to train the convolutional neural network.
  • the parameters of the network are updated, and the result of the previous round of convolution operation of the convolution operation accelerator is used as the input of the next round of convolution operation of the convolution operation accelerator.
  • the input signal generating device is used to apply a voltage of a certain amplitude on the word line electrode connected to the non-volatile memory cell that needs conductance adjustment
  • the output signal generating device is used to apply a voltage of a certain amplitude to the non-volatile memory cell that needs conductance adjustment.
  • a voltage of a certain amplitude is applied to the bit line electrode connected to the volatile memory cell.
  • the present invention provides a convolution operation method based on the above-mentioned convolution operation accelerator, including the following steps:
  • the convolution kernel is split into k convolution kernel units of size 2 ⁇ c by row, and k is a positive integer; K different sub-array units of size (M-2(k-1)) ⁇ N are selected from the word line electrode array and combined with k convolution kernel units of size 2 ⁇ c according to the convolution kernel splitting order.
  • the convolution kernel is divided into k convolution kernel units of 2 ⁇ c size and one convolution kernel unit of 1 ⁇ c size by row; in the words Select k sub-array units with size (M-2(k-1)-1) ⁇ N and one sub-array unit with size (M-2k) ⁇ N in the line electrode array; k sub-array units with size (M -2(k-1)-1) ⁇ N sub-array units correspond to k 2 ⁇ c size convolution kernel units one-to-one according to the convolution kernel splitting order; each 2 ⁇ c size convolution unit
  • the core unit and its N-c copies are stored in the two-layer non-volatile memory cell array between all two adjacent word line electrode layers in the corresponding sub-array unit; for the above size, it is (M-2k) ⁇ N
  • Each word line electrode layer in the sub-array unit stores a 1 ⁇ c size convolution kernel unit and N-c of the 1 ⁇ c size convolution kernel unit in one of the non-vola
  • the two-dimensional input data is converted into corresponding voltage values and applied to the corresponding word line electrodes through the input signal generating device; wherein, the two-dimensionally arranged input data are arranged one by one according to the coordinate information. Corresponding to the two-dimensionally arranged word line electrodes.
  • the dot multiplication operation of the convolution kernel unit and different parts of the two-dimensional input data is implemented in parallel, and the corresponding parts of the convolution kernel unit and the two-dimensional input data are output in parallel through the corresponding bit line electrode layer.
  • the sum of the results of the dot multiplication operation is used to realize the convolution operation of the convolution kernel and the two-dimensional input data;
  • the size of the two-dimensional input data is M ⁇ N; after each row of convolution kernel data of the convolution kernel unit is sequentially stored on the corresponding diagonal line of the corresponding non-volatile storage unit array, it is horizontally moved along the convolution kernel sliding direction. Move to the adjacent N-c diagonal lines and store the row of convolution kernel data again, realizing that each layer of non-volatile storage unit array stores the convolution kernel data of the corresponding row in the convolution kernel unit and the convolution kernel. N-c copies of data.
  • a current is generated on the bit line electrode, and an output signal reading device is used to convert the current on the bit line into a voltage to obtain an output voltage matrix.
  • Figure 5 shows a flow chart of the convolution operation based on the three-dimensional non-volatile memory array described in Embodiment 1, which includes the following steps:
  • pixel values in the range of 0 to 255 are linearly mapped to voltage values within a certain range.
  • This voltage range is selected according to different types of non-volatile memories and follows the principle that the maximum voltage value does not exceed the set voltage of the non-volatile memory.
  • the set voltage of the non-volatile memory is V set
  • the voltage value corresponding to the pixel value is Where P is the pixel value
  • 255 is the maximum value of the pixel value.
  • V set is 1V
  • the mapped voltage range is 0 ⁇ 1V.
  • the voltage value mapped by the pixel value in the first row and first column of the 4 ⁇ 4 picture corresponds to V11 in Figure 3. It is input in the word line electrode corresponding to V11, and the pixel value mapped by the second row and first column is The voltage value corresponds to V12 in Figure 3, and so on, then the 4 ⁇ 4 voltage matrix composed of V11 to V44 in Figure 5 corresponds to the pixels in the 4 ⁇ 4 picture one by one.
  • V41 ⁇ V44 are the voltage values corresponding to the fourth row of input image data
  • k11 ⁇ k22 are the weights of the 2 ⁇ 2 convolution kernel.
  • k11 and k12 are the weights of the first row in the convolution kernel
  • k21 and k22 are the weights of the second row in the convolution kernel.
  • the weight of the second row in the convolution kernel needs to be in the fourth row.
  • k21 and k22 need to slide on V41 ⁇ V44 three times, so k21 and k22 are copied three times and mapped to the positions shown in the upper left picture in Figure 5, respectively corresponding to 421 in Figure 4 and positions 412, 422 and 413, 423 and 414.
  • the convolution kernel mapping methods of other non-volatile memory layers can be deduced in turn.
  • Figure 7 is a side view of a three-dimensional non-volatile memory, showing the mapping scheme of the convolution kernel on other non-volatile memory unit array layers, where V11 ⁇ V41 are the voltage values corresponding to the first column of data in the input image data.
  • V21 as an example
  • multiplication operations need to be performed with k21 and k11 respectively, so the upper and lower non-volatile memory units connected to the word line electrode corresponding to V21 map k21 and k11 respectively.
  • k11 at the same time, the convolution kernel mapping method within each non-volatile storage array layer can be deduced based on Figure 6 and the content described in the previous paragraph. Therefore, each two-layer non-volatile memory cell array completely maps three copies of a 2 ⁇ 2 convolution kernel, and the convolution kernel also needs to slide three times in the y direction, so six layers of non-volatile memory are required.
  • the weight of the non-volatile memory cell is adjusted by applying voltage to the word line and bit line.
  • the voltage on both sides of the non-volatile memory cell is equal to the set voltage.
  • the non-volatile memory cell The voltage on both sides of the memory unit is equal to the reset voltage until it is adjusted to the conductance value corresponding to the convolution kernel weight.
  • V41 is applied to word line 1, and the current is obtained on bit line 2 after passing through k21 according to the current path shown in 1.
  • V42 is applied to word line 2, as shown in 2
  • the current path passes through k22 and gets current on bit line 2.
  • V31 and V32 get current on bit line 2 after passing through k11 and k12 respectively.
  • V42 and V32 participate in the convolution operation because the convolution kernel is in the x direction.
  • the two multiplication and addition operations generated by sliding up realize the reuse of input data.
  • V21 ⁇ V24 and V31 ⁇ V34 will act on the upper and lower layers of non-volatile memory arrays connected to the word line electrodes, and participate in the convolution operation due to the convolution kernel sliding in the y direction.
  • the two multiplication and addition operations generated realize the reuse of input data.
  • the output current matrix is linearly converted into a voltage matrix through the output signal readout device, and this voltage is passed to the next convolution operation as input, or this voltage matrix is stored as the final output of the convolution operation.
  • Figure 8 shows the main view of a three-dimensional non-volatile memory array with eight word lines in the lateral direction, showing the mapping scheme of a 3 ⁇ 3 convolution kernel on a multi-layer non-volatile memory unit layer.
  • the data is 4 ⁇ 4 size.
  • the first, second, and third rows of the accumulation kernel; V1, V2, V3, and V4 correspond to the first, second, third, and fourth rows of the 4 ⁇ 4 input.
  • V1 to V3 are sequentially input to the first, second, and third layer word line electrodes; the first, second, and The third and fourth layers of non-volatile memory unit layers map k1, k2, k1, and k2 respectively.
  • the arrays corresponding to the four word lines on the left complete the convolution operations of k1 and k2.
  • V3 and V4 are input to the first and second layer word line electrodes; the first and third layer non-volatile memory unit layers are mapped k3, the non-volatile memory cells in the second and fourth layers of non-volatile memory unit layers are all set to a high-impedance state, and the arrays corresponding to the four word lines on the left complete the convolution operation of k3.
  • the mapping scheme for each layer is shown in Figure 9.
  • Figure 9 is a top cross-sectional view of a three-dimensional non-volatile memory array with eight word lines in the lateral direction, showing the mapping scheme of a 3 ⁇ 3 convolution kernel on a layer of non-volatile memory cells.
  • V11 ⁇ V14 are the inputs of the first row. Taking the first row of convolution kernels k11 ⁇ k13 and the third row of convolution kernels k31 ⁇ k33 as examples, the mapping scheme on a non-volatile memory unit layer is shown. Input V11 ⁇ V14 on the 4 word lines on the left.
  • map k11 ⁇ k13 on the diagonal line In the non-volatile memory unit layer corresponding to the 4 word lines on the left, map k11 ⁇ k13 on the diagonal line and copy it once in parallel; on the 4 words on the right Input V11 ⁇ V14 on the line, in the non-volatile memory unit layer corresponding to the 4 word lines on the right, map k31 ⁇ k33 on the diagonal line, and copy it once in parallel.
  • the present invention proposes a convolutional neural network operation accelerator and operation method based on a three-dimensional non-volatile memory array.
  • the three-dimensional non-volatile memory array consists of a layer of word line electrodes and a layer of non-volatile memory.
  • the memory unit, a layer of bit line electrodes, and a layer of planar non-volatile memory cell array are stacked in sequence to form a basic unit superposition structure.
  • the word line electrode serves as the input end of the convolution operation and passes through the corresponding non-volatile memory array.
  • a convolution operation is performed, and the output of the convolution operation is obtained after superposition of the current on the bit line.
  • the three-dimensional non-volatile storage array proposed by the present invention has a unique topological structure, which can naturally match the process of the convolution kernel sliding on the input data to perform multiplication and addition operations in the convolution operation, and perform the convolution operation on the two-dimensional input data at one time , to obtain two-dimensional output data.
  • it can not destroy the position information of the input data matrix, and can obtain data output in matrix form, which conforms to the logic of convolution operation in the software.
  • it has high parallelism and high integration, and can occupy a smaller space. Get more operations under the area.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Neurology (AREA)
  • Semiconductor Memories (AREA)
  • Read Only Memory (AREA)

Abstract

一种卷积运算加速器及卷积运算方法，属于微电子器件领域；其中，每条字线电极将矩阵排列的非易失性存储单元中的一列连接起来，每条位线电极倾斜地将矩阵排列的非易失性存储单元连接起来；每条字线的输入数据可以和上下两层卷积核单元进行乘加操作，实现卷积核单元在二维输入中y方向上的天然滑动；倾斜的位线和卷积核在每层非易失性存储阵列中的多次复制可以使一个输入数据和相同卷积核单元中不同位置的卷积核数据进行乘法运算，实现卷积核单元在二维输入中x方向上的天然滑动，可以在不将二维输入展开为一维的情况下一次性并行完成卷积计算，运算速度快，并行程度及阵列单元的利用率均较高。

Description

一种卷积运算加速器及卷积运算方法 【技术领域】
本发明属于微电子器件领域，更具体地，涉及一种卷积运算加速器及卷积运算方法。
【背景技术】
卷积神经网络(Convolution Neural network,CNN)是前馈神经网络的一种形式，通常包含卷积层、池化层和全连接层。卷积神经网络通常用来处理图片信息，将图片输入到卷积层后，经过卷积运算提取出局部特征，然后通过池化层压缩模型大小，减少计算量，最后通过全连接层将特征整合得到输出。卷积神经网络具有局部连接和权值共享的特点，其参数量远远小于多层感知机模型，在图像识别领域已经取得了良好的效果。
卷积神经网络虽然有较少的参数，但在前向传播过程中，卷积层中的卷积运算产生了网络中大部分的计算量，卷积运算过程为：卷积核在输入数据上以一定步长滑动，每滑动一次进行一次卷积核与对应输入数据的乘加运算，这在传统计算单元(例如CPU、GPU)中计算速度慢、占用空间大。
同时对于一些基于新型存储器的计算结构，如平面交叉开关(crossbar)阵列来说，需要将二维的输入信息拉伸为一维然后输入给阵列；另外，如果要充分利用阵列单元，就必须采用滚动式输入，那么输入的并行程度很低，所需时间代价巨大；如果在crossbar阵列中复制多个卷积核进行大规模并行计算的话，会占用大量面积，同时大部分单元不会被使用，增加漏电流对计算结果的影响。
因此，需要一种能够高度并行计算并且能够充分利用计算资源的架构来加速卷积运算。
【发明内容】
针对现有技术的以上缺陷或改进需求，本发明提供了一种卷积运算加速器及卷积运算方法，用以解决平面存储器阵列进行卷积计算时需要将二维输入拉伸为一维、并行程度低以及阵列单元利用率低的问题。
为了实现上述目标，本发明提供了一种卷积运算加速器，包括：三维非易失性存储器阵列和控制模块；
三维非易失性存储器阵列包括：P个字线电极层；任意相邻的两个字线电极层之间均放置有一个位线电极层；任意相邻的字线电极层和位线电极层之间均放置有一个非易失性存储单元阵列，且非易失性存储单元阵列与字线电极层和位线电极层均垂直相连；
字线电极层包括多个平行排布的字线电极；P个字线电极层中的字线电极共同构成一个字线电极阵列；
非易失性存储单元阵列中的每一列非易失性存储单元均连接在与非易失性存储单元阵列相连的字线电极层中的同一条字线上；非易失性存储单元阵列中每一条斜线上的非易失性存储单元均连接在与非易失性存储单元阵列相连的位线电极阵列中的同一条位线上；其中，斜线为非易失性存储单元阵列中与非易失性存储单元阵列中的对应对角线平行的斜线；
记二维输入数据的尺寸为M×N；
当与二维输入数据进行卷积运算的卷积核尺寸为2k×c时，控制模块用于将卷积核按行拆分为k个2×c大小的卷积核单元，k为正整数；在字线电极阵列中选中k个大小为(M-2(k-1))×N的不同子阵列单元，并与k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；将二维输入数据上第2i-1行到第M-2(k-i)行的数据以电压的方式按照对应的坐标信息施加到第i个子阵列单元中的对应字线电极上，i＝1,2,...,k；
当卷积核的尺寸为(2k+1)×c时，控制模块用于将卷积核按行拆分为k个2×c大小的卷积核单元和一个1×c大小的卷积核单元；在字线电极阵列中选中k个大小为(M-2(k-1)-1)×N的子阵列单元，以及一个大小为(M-2k)×N的子阵列单元；k个大小为(M-2(k-1)-1)×N的子阵列单元与k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个2×c大小的卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；对上述大小为(M-2k)×N的子阵列单元中的各字线电极层，在与其相连的其中一个非易失性存储单元阵列中分别存储1×c大小的卷积核单元及该1×c大小的卷积核单元的N-c次复制，未存储卷积核单元的非易失性存储单元阵列中的非易失存储单元全部置为高阻态；将二维输入数据上第2i-1行到第M-2(k-i)-1行的数据以电压的方式按照对应的坐标信息施加到第i个(M-2(k-1)-1)×N大小的子阵列单元中的对应字线电极上，i＝1,2,...,k；将二维输入数据上第2k+1行到第M行的数据以电压的方式按照对应的坐标信息施加到上述大小为(M-2k)×N的子阵列单元中的对应字线电极上；
三维非易失性存储器阵列用于基于非易失性存储单元阵列并行地实现卷积核单元与二维输入数据不同部分的点乘运算，经由对应的位线电极层并行输出卷积核单元与二维输入数据对应部分的点乘运算结果之和，从而实现卷积核与二维输入数据的卷积运算；
其中，卷积核单元的每一行卷积核数据在对应非易失性存储单元阵列的对应斜线上依次存储后，沿着卷积核滑动方向水平移动到相邻的N-c条斜线上再次对该行卷积核数据进行存储，实现每一层非易失性存储单元阵列存储卷积核单元中对应行的卷积核数据及该卷积核数据的N-c次复制。
进一步优选地，当卷积核的尺寸为2k×c时，三维非易失性存储器阵列用于基于上述大小为(M-2(k-1))×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现2×c大小的卷积核单 元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出2×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；控制模块还用于对所选中的各子阵列单元中同一相对位置的位线输出分别进行汇总，得到卷积核与二维输入数据的卷积运算结果；
当卷积核的尺寸为(2k+1)×c时，三维非易失性存储器阵列用于基于上述大小为(M-2(k-1)-1)×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现2×c大小的卷积核单元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出2×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；同时基于上述大小为(M-2k)×N的子阵列单元中的非易失性存储单元阵列，并行地实现1×c大小的卷积核单元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出1×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；控制模块还用于将所选中的各子阵列单元中的各位线输出对应进行汇总，得到卷积核与二维输入数据的卷积运算结果。
进一步优选地，三维非易失性存储器阵列为多层堆叠结构，由字线电极层、非易失性存储单元阵列和位线电极层依次堆叠形成；各字线电极层之间、字线电极层与位线电极层之间均互相平行。
进一步优选地，控制模块包括：输入端信号发生装置、输出端信号发生装置和输出信号读出装置；
输入端信号发生装置和输出端信号发生装置相互配合，在子阵列单元所对应的字线电极和位线电极上施加电压，选择对应的非易失存储单元，并改变所选中的各非易失存储单元的电导值，使其存储对应的卷积核数据；
输入端信号发生装置还用于将二维输入数据以电压的方式，按照二维输入数据的坐标信息施加到上述子阵列单元中的对应字线电极上；
输出信号读出装置用于检测位线电极上的输出电流，并转化为电压信号，作为卷积运算的结果。
进一步优选地，输入信号发生装置的读电压小于或等于非易失存储单元上的当前set电压值，以保证在进行卷积运算时非易失存储单元的电导值不被改变。
进一步优选地，上述卷积运算加速器用于实现卷积神经网络中的卷积运算。
进一步优选地，在卷积神经网络的训练过程中，上述控制模块还用于通过在对应的字线电极和位线电极上施加写电压来调整各非易失存储单元的电导值，以对卷积神经网络的参数进行更新，并将卷积运算加速器上一轮的卷积运算结果重新作为卷积运算加速器下一轮卷积运算的输入。
进一步优选地，当需要增加非易失存储单元的电导值时，在对应的字线电极和位线电极上施加的写电压差值的绝对值为对应的set电压值；
当需要减小非易失存储单元的电导值时，在对应的字线电极和位线电极上施加的写电压差值的绝对值为对应的reset电压值。
第二方面，本发明提供了一种基于上述卷积运算加速器的卷积运算方法，包括以下步骤：
S1、当与二维输入数据进行卷积运算的卷积核尺寸为2k×c时，将卷积核按行拆分为k个2×c大小的卷积核单元，k为正整数；在字线电极阵列中选中k个大小为(M-2(k-1))×N的不同子阵列单元，并与k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；将二维输入数据上第2i-1行到第M-2(k-i)行的数据以电压的方式按照对应的坐标信息施加到第i个子阵列单元中的对应字线电极上，i＝1,2,...,k；
当卷积核的尺寸为(2k+1)×c时，将卷积核按行拆分为k个2×c大小的卷积核单元和一个1×c大小的卷积核单元；在字线电极阵列中选中k个大小为(M-2(k-1)-1)×N的子阵列单元，以及一个大小为(M-2k)×N的子阵 列单元；k个大小为(M-2(k-1)-1)×N的子阵列单元与k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个2×c大小的卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；对上述大小为(M-2k)×N的子阵列单元中的各字线电极层，在与其相连的其中一个非易失性存储单元阵列中分别存储1×c大小的卷积核单元及该1×c大小的卷积核单元的N-c次复制，未存储卷积核单元的非易失性存储单元阵列中的非易失存储单元全部置为高阻态；将二维输入数据上第2i-1行到第M-2(k-i)-1行的数据以电压的方式按照对应的坐标信息施加到第i个(M-2(k-1)-1)×N大小的子阵列单元中的对应字线电极上，i＝1,2,...,k；将二维输入数据上第2k+1行到第M行的数据以电压的方式按照对应的坐标信息施加到上述大小为(M-2k)×N的子阵列单元中的对应字线电极上；
S2、基于非易失性存储单元阵列并行地实现卷积核单元与二维输入数据不同部分的点乘运算，经由对应的位线电极层并行输出卷积核单元与二维输入数据对应部分的点乘运算结果之和，从而实现卷积核与二维输入数据的卷积运算；
其中，二维输入数据的尺寸为M×N；卷积核单元的每一行卷积核数据在对应非易失性存储单元阵列的对应斜线上依次存储后，沿着卷积核滑动方向水平移动到相邻的N-c条斜线上再次对该行卷积核数据进行存储，实现每一层非易失性存储单元阵列存储卷积核单元中对应行的卷积核数据及该卷积核数据的N-c次复制。
进一步优选地，当卷积核的尺寸为2k×c时，步骤S2包括：基于上述大小为(M-2(k-1))×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现2×c大小的卷积核单元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出2×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和，并对所选中的 各子阵列单元中同一相对位置的位线输出分别进行汇总，得到卷积核与二维输入数据的卷积运算结果；
当卷积核的尺寸为(2k+1)×c时，步骤S2包括：基于上述大小为(M-2(k-1)-1)×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现2×c大小的卷积核单元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出2×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；同时基于上述大小为(M-2k)×N的子阵列单元中的非易失性存储单元阵列，并行地实现1×c大小的卷积核单元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出1×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；并将所选中的各子阵列单元中的各位线输出对应进行汇总，得到卷积核与二维输入数据的卷积运算结果。
总体而言，通过本发明所构思的以上技术方案，能够取得以下有益效果：
1、本发明提供了一种卷积运算加速器，其中，每条字线电极将矩阵排列的非易失性存储单元中的一列连接起来，每条位线电极倾斜地将矩阵排列的非易失性存储单元连接起来；每条字线的输入数据可以和上下两层卷积核单元进行乘加操作，实现卷积核单元在二维输入中y方向上的天然滑动；倾斜的位线和卷积核在每层非易失性存储阵列中的多次复制可以使一个输入数据和相同卷积核单元中不同位置的卷积核数据进行乘法运算，实现卷积核单元在二维输入中x方向上的天然滑动，因此可以在不将二维输入展开为一维的情况下一次性并行完成卷积计算，数据复用率和运算的并行程度均较高、运算速度快；与传统平面阵列相比，本发明利用三维阵列的高集成度特性，在较小的面积下容纳更多单元，执行更多次的计算，大大提高了阵列单元的利用率。
2、本发明所提供的卷积运算加速器，分别考虑卷积核行为奇数和偶数的情况进行拆分并存储，能够并行地实现不同拆分部分的与对应二维数据 的卷积运算，充分利用了三维非易失性存储器阵列的计算资源，运算速度较快。
3、本发明所提供的卷积运算加速器可以用来实现卷积神经网络的功能，将神经网络的输入、权重和输出映射为三维非易失性存储器的输入电压、电导以及输出电流，同时由于三维非易失性存储器阵列拓扑结构的独特设计，阵列层面的物理计算过程能够天然匹配卷积神经网络中大量存在的卷积核滑动运算过程，能够在保留二维数据的位置信息的情况下一次性完成卷积运算，并且获得二维输出数据，高度还原了软件中的卷积神经网络的计算过程，同时利用尽可能少的输入输出端口实现卷积高度并行计算，大大提高了资源利用率和运算速度。
4、本发明基于三维非易失性存储器阵列的卷积运算，相比于传统的冯诺依曼架构计算单元例如CPU、GPU等，没有存储器和计算单元之间的数据传输速度限制，实现了存算一体，能够用于实现高效大规模并行的卷积神经网络计算。
【附图说明】
图1为本发明第一方面提供的一种卷积运算加速器的结构示意图；
图2为本发明实施例1提供的卷积运算加速器的结构示意图；
图3为本发明实施例1提供的三维非易失性存储器阵列的结构示意图；
图4为本发明实施例1提供的三维非易失性存储器的俯视剖面图；
图5为本发明实施例2提供的基于实施例1所述三维非易失性存储器阵列的卷积运算的流程图；
图6为本发明实施例2提供的三维非易失性存储器阵列天然匹配卷积神经网络中卷积核在输入数据的x方向上滑动执行乘加运算的示意图；
图7为本发明实施例2提供的三维非易失性存储阵列天然匹配卷积神经网络中卷积核在输入数据的y方向上滑动执行乘加运算的示意图；
图8为本发明实施例3提供的横向上具有八条字线的三维非易失性存储阵列的主视图。
图9为本发明实施例3提供的横向上具有八条字线的三维非易失性存储阵列的俯视剖视图。
【具体实施方式】
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。此外，下面所描述的本发明各个实施方式中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。
为了实现上述目的，如图1所示，本发明提供了一种卷积运算加速器，包括：三维非易失性存储器阵列和控制模块；
其中，三维非易失性存储器阵列包括：P个字线电极层；任意相邻的两个字线电极层之间均放置有一个位线电极层；任意相邻的字线电极层和位线电极层之间均放置有一个非易失性存储单元阵列，且非易失性存储单元阵列与字线电极层和位线电极层均垂直相连；具体地，三维非易失性存储器阵列为多层堆叠结构，由字线电极层、非易失性存储单元阵列和位线电极层依次堆叠形成；各字线电极层之间、字线电极层与位线电极层之间均互相平行；
字线电极层包括多个平行排布的字线电极；P个字线电极层中的字线电极共同构成一个字线电极阵列；
非易失性存储单元阵列中的每一列非易失性存储单元均连接在与非易失性存储单元阵列相连的字线电极层中的同一条字线上；非易失性存储单元阵列中每一条斜线上的非易失性存储单元均连接在与非易失性存储单元阵列相连的位线电极阵列中的同一条位线上；其中，斜线为非易失性存储单 元阵列中与非易失性存储单元阵列中的对应对角线平行的斜线；
记二维输入数据的尺寸为M×N；
当与二维输入数据进行卷积运算的卷积核尺寸为2k×c时，控制模块用于将卷积核按行拆分为k个2×c大小的卷积核单元，k为正整数；在字线电极阵列中选中k个大小为(M-2(k-1))×N的不同子阵列单元，并与k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；将二维输入数据上第2i-1行到第M-2(k-i)行的数据以电压的方式按照对应的坐标信息施加到第i个子阵列单元中的对应字线电极上，i＝1,2,...,k；
当卷积核的尺寸为(2k+1)×c时，控制模块用于将卷积核按行拆分为k个2×c大小的卷积核单元和一个1×c大小的卷积核单元；在字线电极阵列中选中k个大小为(M-2(k-1)-1)×N的子阵列单元，以及一个大小为(M-2k)×N的子阵列单元；k个大小为(M-2(k-1)-1)×N的子阵列单元与k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个2×c大小的卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；对上述大小为(M-2k)×N的子阵列单元中的各字线电极层，在与其相连的其中一个非易失性存储单元阵列中分别存储1×c大小的卷积核单元及该1×c大小的卷积核单元的N-c次复制，未存储卷积核单元的非易失性存储单元阵列中的非易失存储单元全部置为高阻态；将二维输入数据上第2i-1行到第M-2(k-i)-1行的数据以电压的方式按照对应的坐标信息施加到第i个(M-2(k-1)-1)×N大小的子阵列单元中的对应字线电极上，i＝1,2,...,k；将二维输入数据上第2k+1行到第M行的数据以电压的方式按照对应的坐标信息施加到上述大小为(M-2k)×N的子阵列单元中的对应字线电极上；
三维非易失性存储器阵列用于基于非易失性存储单元阵列并行地实现卷积核单元与二维输入数据不同部分的点乘运算，经由对应的位线电极层并行输出卷积核单元与二维输入数据对应部分的点乘运算结果之和，从而实现卷积核与二维输入数据的卷积运算；
其中，卷积核单元的每一行卷积核数据在对应非易失性存储单元阵列的对应斜线上依次存储后，沿着卷积核滑动方向水平移动到相邻的N-c条斜线上再次对该行卷积核数据进行存储，实现每一层非易失性存储单元阵列存储卷积核单元中对应行的卷积核数据及该卷积核数据的N-c次复制。
具体地，当卷积核的尺寸为2k×c时，三维非易失性存储器阵列用于基于上述大小为(M-2(k-1))×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现2×c大小的卷积核单元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出2×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；控制模块还用于对所选中的各子阵列单元中同一相对位置的位线输出分别进行汇总，得到卷积核与二维输入数据的卷积运算结果；
当卷积核的尺寸为(2k+1)×c时，三维非易失性存储器阵列用于基于上述大小为(M-2(k-1)-1)×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现2×c大小的卷积核单元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出2×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；同时基于上述大小为(M-2k)×N的子阵列单元中的非易失性存储单元阵列，并行地实现1×c大小的卷积核单元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出1×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；控制模块还用于将所选中的各子阵列单元中的各位线输出对应进行汇总，得到卷积核与二维输入数据的卷积运算结果。其中，各子阵列单元的输出均为(M-2k)×(N-c)，对应加和后即可以得到最终 的卷积运算结果。
在一种可选实施方式下，当卷积核的尺寸为2k×c时，纵向上至少选中M-2(k-1)层字线层，横向上至少选中k×N条字线；控制模块将卷积核拆分为k个2×c大小的卷积核单元，将横向上有k×N个字线上的三维非易失性存储阵列视为k个具有N个字线的子阵列单元，在每一个子阵列单元的对应位置映射一个2×c大小的卷积核单元，二维输入数据在k个阵列单元上同样也会滑动，例如，M×N个输入数据，第一个阵列单元字线上映射第一行到第M-2(k-1)行数据，第二个阵列单元字线上映射第2+1行到M-2(k-2)行数据，第i个阵列单元字线上映射第2i-1行到第M-2(k-i)(i∈1,2,3,…,k)行数据。此时，控制模块还用于对k个子阵列单元的中同一相对位置的位线输出分别进行汇总，得到最终的卷积运算结果。
当卷积核的尺寸为(2k+1)×c时，k个2×c大小的卷积核的映射方式和卷积核的尺寸为2k×c时的方法类似，区别在于，在横向上选中(k+1)×N个字线的三维非易失存储阵列，将横向上有(k+1)×N个字线上的三维非易失性存储阵列视为k+1个具有N个字线的子阵列单元，第i个阵列单元字线上映射第2i-1行到第M-2(k-i)-1(i∈1,2,3,…,k)行数据；对于剩余的一个1×c大小的卷积核单元，将其映射到第k+1个阵列单元上，对于每层位线连接的两层非易失性存储单元，将其上方(或下方)的非易失性存储单元阵列中存储1×c大小的卷积核单元及其N-c次复制，将位线下方(或上方)的非易失性存储单元全部置为高阻态(视为断路)；本具体实施方式在纵向上选中M-2(k-1)-1层字线层，横向上选中(k+1)×N条字线；当在上方的非易失性存储单元阵列上存储1×c大小的卷积核单元时，在从上至下数第1层到第M-2k层上映射二维输入数据中的第2k+1行到第M行数据。当在下方的非易失性存储单元阵列上存储1×c大小的卷积核单元时，在从上至下数第2层到第M-2k+1层上映射二维输入数据中的第2k+1行到第M行数据。此时，控制模块还用于对k+1个子阵列单元的中同一相对位置的位线输出分别进行 汇总，得到最终的卷积运算结果。
为了进一步说明本发明所提供的卷积运算加速器，下面结合具体实施例进行详述：
实施例1、
本实施例中的卷积运算加速器如图2所示，包括：三维非易失性存储器阵列和控制模块，其中，控制模块包括：输入端信号发生装置、输出端信号发生装置和输出信号读出装置；
如图3所示为三维非易失性存储器阵列的结构示意图，包括字线电极1、位线电极2、非易失存储单元3。输入端信号发生装置和字线电极相连，输出端信号发生装置和输出信号读出装置连接至位线电极。V11～V44以电压形式施加到字线电极上作为输入，I11～I33为输入电压作用到非易失性存储阵列后得到的输出电流。
为了进一步说明三维非易失存储器阵列内部的连接结构，如图4展示了三维非易失性存储器的俯视剖面图。左边所示的俯视剖面图为字线电极和非易失性存储单元阵列的两层结构，非易失存储单元阵列在字线电极上方。字线1～4为平行排布的电极，每条字线上各自均匀分布着非易失性存储单元，非易失性存储单元411～441连接至字线1，非易失性存储单元412～442连接至字线2，非易失性存储单元413～443连接至字线3，非易失性存储单元414～444连接至字线4。右边所示的俯视剖面图在左边的俯视剖面图基础上添加了一层位线电极，自下而上分别为字线电极、非易失性存储器阵列和位线电极。位线电极平行排布，但与字线电极的方向呈一定角度。非易失性存储单元411连接至位线1，非易失性存储单元421、412连接至位线2，非易失性存储单元431、422、413连接至位线3，非易失性存储单元441、432、423、414连接至位线4。
需要说明的是，在三维非易失性存储器阵列中，每条字线将矩阵排列的非易失性存储单元中的一列连接起来，每条位线倾斜地将矩阵排列的非易 失性存储单元连接起来。每条字线的输入数据可以和上下两层卷积核单元进行乘加操作，实现卷积核单元在二维输入中y方向上的天然滑动。倾斜的位线和卷积核在每层非易失性存储阵列中的多次复制可以使一个输入数据和相同卷积核中不同位置的卷积核数据进行乘法运算，实现卷积核在二维输入中x方向上的天然滑动，因此可以在不将二维输入展开为一维的情况下一次性并行完成卷积计算，运算速度极快。
进一步地，输入端信号发生装置和输出端信号发生装置相互配合，在子阵列单元所对应的字线电极和位线电极上施加电压，选择对应的非易失存储单元，并改变所选中的各非易失存储单元的电导值，使其存储对应的卷积核数据；
输入端信号发生装置还用于将二维输入数据以电压的方式，按照二维输入数据的坐标信息施加到上述子阵列单元中的对应字线电极上；
输出信号读出装置用于检测位线电极上的输出电流，并转化为电压信号，作为卷积运算的结果。
具体地，在卷积运算过程中，输入端信号发生装置用于将输入数据对应的电压施加在对应的字线电极上，且此时电压幅值不超过非易失存储单元的set电压。输出信号读出装置用于将位线电极上的输出电流转化为电压，并读出此电压值，作为卷积运算的结果。
需要说明的是，上述卷积运算加速器用于实现卷积神经网络中的卷积运算。此时，三维非易失性存储器阵列的字线电极作为卷积神经网络的输入，每层字线电极对应卷积神经网络二维输入中的一行，每层非易失存储阵列对应卷积核中的一行，每层位线电极对应着神经网络二维输出中的一行。输入端信号发生装置用于在字线电极上施加电压，所施加的电压有两个用途，一是用电压表示输入数据，经过调整好的电导值得到电流输出，完成神经网络的前向传播运算；二在字线电极上施加电压并配合位线电极来选中一个非易失性存储单元，并改变选中的非易失存储单元的电导值。输出端信号发 生装置用于在位线电极上施加电压，配合字线电极选中一个非易失性存储单元，并改变选中的非易失性存储单元的电导值。输出信号读出装置用于检测位线电极上的输出电流，并转化为电压信号，作为卷积运算的结果。另外，在卷积神经网络的训练过程中，上述控制模块还用于通过在对应的字线电极和位线电极上施加写电压来调整各非易失存储单元的电导值，以对卷积神经网络的参数进行更新，并将卷积运算加速器上一轮的卷积运算结果重新作为卷积运算加速器下一轮卷积运算的输入。具体地，输入端信号发生装置用于在与需要进行电导调节的非易失性存储单元连接的字线电极上施加一定幅值的电压，同时输出端信号发生装置在与需要进行电导调节的非易失性存储单元连接的位线电极上施加一定幅值的电压，当一个非易失性存储单元两端的字线电极和位线电极都被施加了电压后，则代表此单元被选中，而输入端信号发生装置或输出端信号发生装置单独施加的电压不能改变非易失存储单元的电导值；输入端信号发生装置和输出端信号发生装置施加的写电压相减之后的绝对值需等于set电压值(置态电压)或者reset电压值(重置电压)，取决于在更新过程中电导值需要被增加或者减少；此时，单元两端电压差等于此单元的set电压或者reset电压；具体地，需要增加电导时，非易失性存储单元两侧电压等于set电压；需要减少电导时，非易失性存储单元两侧电压等于reset电压，直至调节为卷积核权重对应的电导值。
第二方面，本发明提供了基于上述卷积运算加速器的卷积运算方法，包括以下步骤：
S1、当与二维输入数据进行卷积运算的卷积核尺寸为2k×c时，将卷积核按行拆分为k个2×c大小的卷积核单元，k为正整数；在字线电极阵列中选中k个大小为(M-2(k-1))×N的不同子阵列单元，并与k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非 易失性存储单元阵列中；将二维输入数据上第2i-1行到第M-2(k-i)行的数据以电压的方式按照对应的坐标信息施加到第i个子阵列单元中的对应字线电极上，i＝1,2,...,k；
当卷积核的尺寸为(2k+1)×c时，将卷积核按行拆分为k个2×c大小的卷积核单元和一个1×c大小的卷积核单元；在字线电极阵列中选中k个大小为(M-2(k-1)-1)×N的子阵列单元，以及一个大小为(M-2k)×N的子阵列单元；k个大小为(M-2(k-1)-1)×N的子阵列单元与k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个2×c大小的卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；对上述大小为(M-2k)×N的子阵列单元中的各字线电极层，在与其相连的其中一个非易失性存储单元阵列中分别存储1×c大小的卷积核单元及该1×c大小的卷积核单元的N-c次复制，未存储卷积核单元的非易失性存储单元阵列中的非易失存储单元全部置为高阻态；将二维输入数据上第2i-1行到第M-2(k-i)-1行的数据以电压的方式按照对应的坐标信息施加到第i个(M-2(k-1)-1)×N大小的子阵列单元中的对应字线电极上，i＝1,2,...,k；将二维输入数据上第2k+1行到第M行的数据以电压的方式按照对应的坐标信息施加到上述大小为(M-2k)×N的子阵列单元中的对应字线电极上；
具体地，通过在字线和位线选中非易失存储单元，在字线和位线上施加电压增加或者减少选中的非易失存储单元的电导值；在上述过程中，比较被选中单元的电导值和目标电导值，若不相等，则通过字线和位线在该单元两端施加电压脉冲使其电导值和目标电导值相等，最终目标为将非易失存储单元的电导值调节为对应的卷积核数据。
进一步地，在输入二维数据时，将二维输入数据转化为对应电压值，并通过输入信号发生装置施加到对应的字线电极上；其中，二维排布的输入数 据按照坐标信息一一对应到二维排布的字线电极上。
S2、基于非易失性存储单元阵列并行地实现卷积核单元与二维输入数据不同部分的点乘运算，经由对应的位线电极层并行输出卷积核单元与二维输入数据对应部分的点乘运算结果之和，从而实现卷积核与二维输入数据的卷积运算；
其中，二维输入数据的尺寸为M×N；卷积核单元的每一行卷积核数据在对应非易失性存储单元阵列的对应斜线上依次存储后，沿着卷积核滑动方向水平移动到相邻的N-c条斜线上再次对该行卷积核数据进行存储，实现每一层非易失性存储单元阵列存储卷积核单元中对应行的卷积核数据及该卷积核数据的N-c次复制。
具体地，施加在字线电极上的电压作用在非易失性存储阵列后，在位线电极上产生电流，使用输出信号读出装置将位线上的电流转化为电压，得到输出电压矩阵。
为了进一步说明本发明所提供的卷积运算方法，下面结合具体实施例进行详述：
实施例2、
为进一步说明本发明提出的基于三维非易失性存储器阵列的卷积运算方法以及其优势，下面以4×4大小的图片作为输入信息，2×2大小的卷积核为例，阐述本发明提出的卷积运算方法的技术细节。
图5展示了基于实施例1所述三维非易失性存储器阵列的卷积运算的流程图，包含以下步骤：
A1、将4×4大小的图片中共16个像素值映射为电压值，并确定映射后的电压值对应的字线电极；
具体的，将0～255范围内的像素值线性映射到一定范围内的电压值，此电压范围根据不同类型的非易失存储器选择，遵循最大电压值不超过非易失存储器的set电压原则。设非易失性存储器的set电压为V set，则像素值 对应的电压值为
Figure PCTCN2022087794-appb-000001
其中P为像素值，255为像素值的最大值，例如V set为1V，则映射的电压范围在0～1V，当像素值为127.5时，此像素值映射后的电压值为
Figure PCTCN2022087794-appb-000002
具体的，4×4图片中的第一行第一列的像素值映射的电压值对应图3中的V11，在V11对应的字线电极中输入，第二行第一列的像素值映射的电压值对应图3中的V12，依此类推，则图5中V11～V44构成的4×4电压矩阵一一对应着4×4图片中的像素点。
A2、将2×2的卷积核权重映射到非易失性存储阵列上，即分配卷积核权重在非易失性存储阵列上的位置；
具体的，以图6中一层非易失性存储器阵列的映射方式为例，V41～V44对输入图片数据的第四行数据对应的电压值，k11～k22为2×2卷积核的权重，其中k11、k12为卷积核中第一行的权重，k21、k22为卷积核中第二行的权重，按照卷积运算规则，卷积核中第二行的权重需要在第四行的输入数据上滑动进行乘加运算，对应着k21、k22需要在V41～V44上滑动三次，于是将k21和k22复制三次映射到图5中左上图所示的位置，分别对应着图4中421和412、422和413、423和414位置，其他非易失性存储器层的卷积核映射方式可以依次类推。
图7为三维非易失性存储器的侧视图，展示了卷积核在其他非易失存储单元阵列层上的映射方案，其中V11～V41为输入图片数据的第一列数据对应的电压值，以V21为例，在卷积运算中卷积核的滑动过程中，需要分别和k21、k11进行乘法运算，所以V21对应的字线电极相连的上下两个非易失性存储单元分别映射k21和k11，同时每个非易失性存储阵列层内的卷积核映射方式可以结合图6以及上段所述内容类推。所以，每两层非易失存储单元阵列完整映射一个2×2的卷积核的三次复制，而在y方向上卷积核同样需要滑动三次，所以需要六层非易失存储器。
A3、根据卷积核权重值调节非易失性存储器阵列的电导值；
具体的，通过在字线和位线上施加电压调整非易失性存储单元的权重，需要增加电导时，非易失性存储单元两侧电压等于set电压，需要减少电导时，非易失性存储单元两侧电压等于reset电压，直至调节为卷积核权重对应的电导值。
A4、将4×4大小的电压矩阵输入到对应字线电极后，经过非易失性存储阵列在位线电极上得到2×2大小的输出电流矩阵，即由V11～V44得到I11～I33；
具体的，如图6中左下图所示，V41施加在字线1上，按照①所示的电流通路经过k21后在位线2上得到电流，V42施加在字线2上，按照②所示的电流通路经过k22后在位线2上得到电流，同理在图7中，V31和V32分别经过k11和k12后在位线2上得到电流，四股电流在位线2上汇聚为I31，即I31＝V31×k11+V32×k12+V41×k21+V42×k22。同时，V42还会经过k21在位线3上产生电流，即I32＝V32×k11+V33×k12+V42×k21+V43×k22，V42和V32参与了卷积运算中由于卷积核在x方向上滑动而产生的两次乘加运算，实现了输入数据的重复利用。同时在图3和图7中，V21～V24和V31～V34会作用于字线电极相连的上下两层非易失性存储器阵列，参与了卷积运算中由于卷积核在y方向上滑动而产生的两次乘加运算，实现了输入数据的重复利用。4×4的电压矩阵经过复制了多个2×2卷积核的电导阵列后，得到3×3的输出电流矩阵。
A5、输出电流矩阵经过输出信号读出装置线性转化为电压矩阵，将此电压传递给下一次卷积运算作为输入，或者储存此电压矩阵作为卷积运算的最终输出。
实施例3、
为进一步说明本发明提出的基于三维非易失性存储器阵列的卷积运算方法以及其优势，下面以3×3大小的卷积核为例，阐述本发明提出的卷积 运算方法的技术细节。
如图8所示为横向上具有八条字线的三维非易失性存储阵列的主视图，展示了3×3大小的卷积核在多层非易失性存储单元层上的映射方案，输入数据为4×4大小。将3×3的卷积核切分为2×3和1×3的部分分别进行卷积运算，即图8中下方k1，k2和k3，k1、k2、k3分别对应着3×3大小卷积核的第一行、第二行和第三行；V1、V2、V3、V4对应4×4输入的第一行、第二行、第三行和第四行。将V1～V3以及k1、k2映射到图8上方左边四条字线对应的阵列，V1～V3依次输入到第一层、第二层、第三层字线电极；第一层、第二层、第三层、第四层非易失存储单元层分别映射k1、k2、k1、k2，左边四条字线对应的阵列完成k1、k2的卷积运算操作。将V3、V4以及k3映射到图8上方右边四条字线对应的阵列，V3、V4输入到第一层、第二层字线电极；第一层、第三层非易失存储单元层都映射k3，第二层、第四层非易失存储单元层的非易失性存储单元全部置为高阻态，左边四条字线对应的阵列完成k3的卷积运算操作。每层的映射方案在图9中展示。
图9为横向上具有八条字线的三维非易失性存储阵列的俯视剖视图，展示了3×3大小的卷积核在一层非易失性存储单元层上的映射方案。V11～V14为第一行的输入，同时以第一行卷积核k11～k13以及第三行卷积核k31～k33为例，展示在一层非易失性存储单元层上的映射方案。在左边4条字线上输入V11～V14，在左边4条字线对应的非易失性存储单元层中，在斜对角线上映射k11～k13，并平行复制一次；在右边4条字线上输入V11～V14，在右边4条字线对应的非易失性存储单元层中，在斜对角线上映射k31～k33，并平行复制一次。
相关技术方案同实施例1和实施例2，这里不做赘述。
综上所述，本发明提出了一种基于三维非易失性存储器阵列的卷积神经网络运算加速器及运算方法，三维非易失性存储器阵列由一层字线电极、一层非易失性存储单元、一层位线电极、一层平面非易失性存储单元阵列依 次堆叠而形成的基本单元叠加构造而成，字线电极作为卷积运算的输入端，经过对应的非易失存储器阵列进行卷积运算，在位线上进行电流的叠加后得到卷积运算的输出。本发明提出的三维非易失性存储阵列有独特的拓扑结构，能够天然匹配卷积运算中卷积核在输入数据上滑动进行乘加运算的过程，一次性对二维输入数据进行卷积运算，得到二维输出数据。与现有技术相比，能够不破坏输入数据矩阵的位置信息，且能够得到矩阵形式的数据输出，符合软件中进行卷积运算的逻辑，同时并行度高，集成度高，在更小的占用面积下获得更多的运算次数。
本领域的技术人员容易理解，以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种卷积运算加速器，其特征在于，包括：三维非易失性存储器阵列和控制模块；
    所述三维非易失性存储器阵列包括：P个字线电极层；任意相邻的两个字线电极层之间均放置有一个位线电极层；任意相邻的字线电极层和位线电极层之间均放置有一个非易失性存储单元阵列，且非易失性存储单元阵列与字线电极层和位线电极层均垂直相连；
    所述字线电极层包括多个平行排布的字线电极；所述P个字线电极层中的字线电极共同构成一个字线电极阵列；
    非易失性存储单元阵列中的每一列非易失性存储单元均连接在与非易失性存储单元阵列相连的字线电极层中的同一条字线上；非易失性存储单元阵列中每一条斜线上的非易失性存储单元均连接在与非易失性存储单元阵列相连的位线电极阵列中的同一条位线上；所述斜线为非易失性存储单元阵列中与非易失性存储单元阵列中的对应对角线平行的斜线；
    记二维输入数据的尺寸为M×N；
    当与所述二维输入数据进行卷积运算的卷积核尺寸为2k×c时，所述控制模块用于将所述卷积核按行拆分为k个2×c大小的卷积核单元，k为正整数；在字线电极阵列中选中k个大小为(M-2(k-1))×N的不同子阵列单元，并与所述k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；将所述二维输入数据上第2i-1行到第M-2(k-i)行的数据以电压的方式按照对应的坐标信息施加到第i个子阵列单元中的对应字线电极上，i＝1,2,…,k；
    当与所述二维输入数据进行卷积运算的卷积核尺寸为(2k+1)×c时，所述控制模块用于将卷积核按行拆分为k个2×c大小的卷积核单元和一个 1×c大小的卷积核单元；在字线电极阵列中选中k个大小为(M-2(k-1)-1)×N的子阵列单元，以及一个大小为(M-2k)×N的子阵列单元；所述k个大小为(M-2(k-1)-1)×N的子阵列单元与所述k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个2×c大小的卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；对所述大小为(M-2k)×N的子阵列单元中的各字线电极层，在与其相连的其中一个非易失性存储单元阵列中分别存储所述1×c大小的卷积核单元及所述1×c大小的卷积核单元的N-c次复制，未存储卷积核单元的非易失性存储单元阵列中的非易失存储单元全部置为高阻态；将所述二维输入数据上第2i-1行到第M-2(k-i)-1行的数据以电压的方式按照对应的坐标信息施加到第i个(M-2(k-1)-1)×N大小的子阵列单元中的对应字线电极上，i＝1,2,…,k；将所述二维输入数据上第2k+1行到第M行的数据以电压的方式按照对应的坐标信息施加到所述大小为(M-2k)×N的子阵列单元中的对应字线电极上；
    所述三维非易失性存储器阵列用于基于非易失性存储单元阵列并行地实现卷积核单元与二维输入数据不同部分的点乘运算，经由对应的位线电极层并行输出卷积核单元与二维输入数据对应部分的点乘运算结果之和，从而实现所述卷积核与所述二维输入数据的卷积运算；
    其中，卷积核单元的每一行卷积核数据在对应非易失性存储单元阵列的对应斜线上依次存储后，沿着卷积核滑动方向水平移动到相邻的N-c条斜线上再次对该行卷积核数据进行存储，实现每一层非易失性存储单元阵列存储卷积核单元中对应行的卷积核数据及该卷积核数据的N-c次复制。
  2. 根据权利要求1所述的卷积运算加速器，其特征在于，包括：当所述卷积核的尺寸为2k×c时，所述三维非易失性存储器阵列用于基于所述大小为(M-2(k-1))×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现所述2×c大小的卷积核单元与二维输 入数据不同部分的点乘运算后，经由对应的位线电极层并行输出所述2×c大小的卷积核单元与所述二维输入数据对应部分的点乘运算结果之和；所述控制模块还用于对所选中的各子阵列单元中同一相对位置的位线输出分别进行汇总，得到卷积核与二维输入数据的卷积运算结果；
    当所述卷积核的尺寸为(2k+1)×c时，所述三维非易失性存储器阵列用于基于所述大小为(M-2(k-1)-1)×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现2×c大小的卷积核单元与所述二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出所述2×c大小的卷积核单元与所述二维输入数据对应部分的点乘运算结果之和；同时基于所述大小为(M-2k)×N的子阵列单元中的非易失性存储单元阵列，并行地实现所述1×c大小的卷积核单元与所述二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出1×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；所述控制模块还用于将所选中的各子阵列单元中的各位线输出对应进行汇总，得到卷积核与二维输入数据的卷积运算结果。
  3. 根据权利要求1所述的卷积运算加速器，其特征在于，所述三维非易失性存储器阵列为多层堆叠结构，由字线电极层、所述非易失性存储单元阵列和位线电极层依次堆叠形成；各字线电极层之间、字线电极层与位线电极层之间均互相平行。
  4. 根据权利要求1所述的卷积运算加速器，其特征在于，所述控制模块包括：输入端信号发生装置、输出端信号发生装置和输出信号读出装置；
    所述输入端信号发生装置和输出端信号发生装置相互配合，在子阵列单元所对应的字线电极和位线电极上施加电压，选择对应的非易失存储单元，并改变所选中的各非易失存储单元的电导值，使其存储对应的卷积核数据；
    所述输入端信号发生装置还用于将二维输入数据以电压的方式，按照 二维输入数据的坐标信息施加到子阵列单元中的对应字线电极上；
    所述输出信号读出装置用于检测位线电极上的输出电流，并转化为电压信号，作为卷积运算的结果。
  5. 根据权利要求4所述的卷积运算加速器，其特征在于，所述输入信号发生装置的读电压小于或等于非易失存储单元上的当前set电压值，以保证在进行卷积运算时非易失存储单元的电导值不被改变。
  6. 根据权利要求1-5任意一项所述的卷积运算加速器，其特征在于，所述卷积运算加速器用于实现卷积神经网络中的卷积运算。
  7. 根据权利要求6所述的卷积运算加速器，其特征在于，在卷积神经网络的训练过程中，所述控制模块还用于通过在对应的字线电极和位线电极上施加写电压来调整各非易失存储单元的电导值，以对卷积神经网络的参数进行更新，并将卷积运算加速器上一轮的卷积运算结果重新作为卷积运算加速器下一轮卷积运算的输入。
  8. 根据权利要求7所述的卷积运算加速器，其特征在于，当需要增加非易失存储单元的电导值时，在对应的字线电极和位线电极上施加的写电压差值的绝对值为对应的set电压值；
    当需要减小非易失存储单元的电导值时，在对应的字线电极和位线电极上施加的写电压差值的绝对值为对应的reset电压值。
  9. 一种基于权利要求1-8任意一项所述卷积运算加速器的卷积运算方法，其特征在于，包括以下步骤：
    S1、当与所述二维输入数据进行卷积运算的卷积核尺寸为2k×c时，将所述卷积核按行拆分为k个2×c大小的卷积核单元，k为正整数；在字线电极阵列中选中k个大小为(M-2(k-1))×N的不同子阵列单元，并与所述k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；将所述二维输入数据上第2i-1行到第 M-2(k-i)行的数据以电压的方式按照对应的坐标信息施加到第i个子阵列单元中的对应字线电极上，i＝1,2,…,k；
    当与所述二维输入数据进行卷积运算的卷积核尺寸为(2k+1)×c时，将卷积核按行拆分为k个2×c大小的卷积核单元和一个1×c大小的卷积核单元；在字线电极阵列中选中k个大小为(M-2(k-1)-1)×N的子阵列单元，以及一个大小为(M-2k)×N的子阵列单元；所述k个大小为(M-2(k-1)-1)×N的子阵列单元与所述k个2×c大小的卷积核单元按照卷积核的拆分顺序一一对应；将每个2×c大小的卷积核单元及其N-c次复制均存储至对应子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列中；对所述大小为(M-2k)×N的子阵列单元中的各字线电极层，在与其相连的其中一个非易失性存储单元阵列中分别存储所述1×c大小的卷积核单元及所述1×c大小的卷积核单元的N-c次复制，未存储卷积核单元的非易失性存储单元阵列中的非易失存储单元全部置为高阻态；将所述二维输入数据上第2i-1行到第M-2(k-i)-1行的数据以电压的方式按照对应的坐标信息施加到第i个(M-2(k-1)-1)×N大小的子阵列单元中的对应字线电极上，i＝1,2,…,k；将所述二维输入数据上第2k+1行到第M行的数据以电压的方式按照对应的坐标信息施加到所述大小为(M-2k)×N的子阵列单元中的对应字线电极上；
    S2、基于非易失性存储单元阵列并行地实现卷积核单元与二维输入数据不同部分的点乘运算，经由对应的位线电极层并行输出卷积核单元与二维输入数据对应部分的点乘运算结果之和，从而实现所述卷积核与所述二维输入数据的卷积运算；
    其中，所述二维输入数据的尺寸为M×N；卷积核单元的每一行卷积核数据在对应非易失性存储单元阵列的对应斜线上依次存储后，沿着卷积核滑动方向水平移动到相邻的N-c条斜线上再次对该行卷积核数据进行存储， 实现每一层非易失性存储单元阵列存储卷积核单元中对应行的卷积核数据及该卷积核数据的N-c次复制。
  10. 根据权利要求9所述的卷积运算方法，其特征在于，当所述卷积核的尺寸为2k×c时，所述步骤S2包括：基于所述大小为(M-2(k-1))×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现所述2×c大小的卷积核单元与二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出所述2×c大小的卷积核单元与所述二维输入数据对应部分的点乘运算结果之和；对所选中的各子阵列单元中同一相对位置的位线输出分别进行汇总，得到所述卷积核与所述二维输入数据的卷积运算结果；
    当所述卷积核的尺寸为(2k+1)×c时，所述步骤S2包括：基于所述大小为(M-2(k-1)-1)×N的子阵列单元中所有相邻的两个字线电极层之间的两层非易失性存储单元阵列，并行地实现2×c大小的卷积核单元与所述二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出所述2×c大小的卷积核单元与所述二维输入数据对应部分的点乘运算结果之和；同时基于所述大小为(M-2k)×N的子阵列单元中的非易失性存储单元阵列，并行地实现所述1×c大小的卷积核单元与所述二维输入数据不同部分的点乘运算后，经由对应的位线电极层并行输出1×c大小的卷积核单元与二维输入数据对应部分的点乘运算结果之和；将所选中的各子阵列单元中的各位线输出对应进行汇总，得到所述卷积核与所述二维输入数据的卷积运算结果。
PCT/CN2022/087794 2022-03-18 2022-04-20 一种卷积运算加速器及卷积运算方法 Ceased WO2023173530A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/266,610 US12293804B2 (en) 2022-03-18 2022-04-20 Convolution operation accelerator and convolution operation method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210272801.8 2022-03-18
CN202210272801.8A CN114724595B (zh) 2022-03-18 2022-03-18 一种卷积运算加速器及卷积运算方法

Publications (1)

Publication Number Publication Date
WO2023173530A1 true WO2023173530A1 (zh) 2023-09-21

Family

ID=82238502

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/087794 Ceased WO2023173530A1 (zh) 2022-03-18 2022-04-20 一种卷积运算加速器及卷积运算方法

Country Status (3)

Country Link
US (1) US12293804B2 (zh)
CN (1) CN114724595B (zh)
WO (1) WO2023173530A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115348432B (zh) * 2022-08-15 2024-05-07 上海壁仞科技股份有限公司 数据处理方法及装置、图像处理方法、电子设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530210A (zh) * 2016-10-31 2017-03-22 北京大学 基于阻变存储器件阵列实现并行卷积计算的设备和方法
CN106847335A (zh) * 2016-12-27 2017-06-13 北京大学 基于阻变存储阵列的卷积计算存储一体化设备及方法
CN107368889A (zh) * 2017-06-01 2017-11-21 北京大学 基于阻变存储器三维交叉阵列的卷积、池化和激活电路
US20190108437A1 (en) * 2017-10-10 2019-04-11 Fu-Chang Hsu Two and three-dimensional neural network arrays
CN110826709A (zh) * 2019-10-18 2020-02-21 华中科技大学 一种基于三维相变存储器的3d卷积运算装置及方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018137177A1 (zh) * 2017-01-25 2018-08-02 北京大学 一种基于nor flash阵列的卷积运算方法
CN106843809B (zh) * 2017-01-25 2019-04-30 北京大学 一种基于nor flash阵列的卷积运算方法
US10824938B2 (en) * 2017-04-24 2020-11-03 Intel Corporation Specialized fixed function hardware for efficient convolution
US10186011B2 (en) * 2017-04-28 2019-01-22 Intel Corporation Programmable coarse grained and sparse matrix compute hardware with advanced scheduling
CN110210610B (zh) * 2018-03-27 2023-06-20 腾讯科技（深圳）有限公司 卷积计算加速器、卷积计算方法及卷积计算设备
CN111048135A (zh) * 2018-10-14 2020-04-21 天津大学青岛海洋技术研究院 一种基于忆阻器内存计算的cnn处理装置及其工作方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530210A (zh) * 2016-10-31 2017-03-22 北京大学 基于阻变存储器件阵列实现并行卷积计算的设备和方法
CN106847335A (zh) * 2016-12-27 2017-06-13 北京大学 基于阻变存储阵列的卷积计算存储一体化设备及方法
CN107368889A (zh) * 2017-06-01 2017-11-21 北京大学 基于阻变存储器三维交叉阵列的卷积、池化和激活电路
US20190108437A1 (en) * 2017-10-10 2019-04-11 Fu-Chang Hsu Two and three-dimensional neural network arrays
CN110826709A (zh) * 2019-10-18 2020-02-21 华中科技大学 一种基于三维相变存储器的3d卷积运算装置及方法

Also Published As

Publication number Publication date
US12293804B2 (en) 2025-05-06
CN114724595A (zh) 2022-07-08
CN114724595B (zh) 2023-03-10
US20250022490A1 (en) 2025-01-16

Similar Documents

Publication Publication Date Title
KR102752652B1 (ko) 메모리내 승산 누적 연산들을 위한 ｎａｎｄ 블록 아키텍처
TWI815392B (zh) 存內計算裝置及用於執行mac操作之方法
US11989646B2 (en) Neuromorphic apparatus having 3D stacked synaptic structure and memory device having the same
CN110826709B (zh) 一种基于三维相变存储器的3d卷积运算装置及方法
US20240160689A1 (en) Method for optimizing convolution operation of system on chip and related product
CN107918794A (zh) 基于计算阵列的神经网络处理器
CN108053848A (zh) 电路结构及神经网络芯片
CN113449612B (zh) 一种基于子流型稀疏卷积的三维目标点云识别的方法
CN110580519A (zh) 一种卷积运算结构及其方法
CN109446478A (zh) 一种基于迭代和可重构方式的复协方差矩阵计算系统
CN114724595B (zh) 一种卷积运算加速器及卷积运算方法
KR20190114208A (ko) 저전력 및 고속 연산을 위한 ｄｒａｍ용 비트와이즈 컨볼루션 회로
CN114168903B (zh) 一种基于ReRAM Crossbar的矩阵乘法计算设备和匹配的方法
CN112598122B (zh) 一种基于可变电阻式随机存储器的卷积神经网络加速器
CN115965067A (zh) 一种针对ReRAM的神经网络加速器
Guo et al. Efficient convolutional networks learning through irregular convolutional kernels
CN109447839B (zh) 非零元素的随机生成及形成电力系统雅可比矩阵的方法
WO2025080293A2 (en) 3d processor
CN119512500A (zh) 一种支持多模卷积的存算一体加速宏、芯片及设备
CN118503205A (zh) 用于处理张量数据的方法和装置
CN113487020B (zh) 用于神经网络计算的参差存储结构及神经网络计算方法
TWI919552B (zh) 記憶體電路及其操作方法
Ko et al. Efficient implementation of multi-channel convolution in monolithic 3D ReRAM crossbar
US20250232163A1 (en) Memory circuits with multi-row storage cells and methods for operating the same
CN115910136B (zh) 一种基于电阻式存储器的卷积加速运算阵列及其控制方法

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 18266610

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22931557

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22931557

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 18266610

Country of ref document: US