WO2021223528A1 - 用于执行卷积神经网络运算的处理装置与处理方法 - Google Patents

用于执行卷积神经网络运算的处理装置与处理方法 Download PDF

Info

Publication number
WO2021223528A1
WO2021223528A1 PCT/CN2021/082078 CN2021082078W WO2021223528A1 WO 2021223528 A1 WO2021223528 A1 WO 2021223528A1 CN 2021082078 W CN2021082078 W CN 2021082078W WO 2021223528 A1 WO2021223528 A1 WO 2021223528A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
weight data
memory
internal memory
processing device
Prior art date
Application number
PCT/CN2021/082078
Other languages
English (en)
French (fr)
Inventor
程韦翰
Original Assignee
神盾股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 神盾股份有限公司 filed Critical 神盾股份有限公司
Publication of WO2021223528A1 publication Critical patent/WO2021223528A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present disclosure relates to a computing device, and more particularly to a processing device and processing method for performing convolutional neural network operations.
  • the processing chip In order to speed up the processing speed and reduce the power consumption caused by repeated access to the external memory, the processing chip is generally provided with an internal memory (also called on-chip memory (on -chip-memory)) to store the temporary calculation results and the weight data required for the convolution operation.
  • this internal memory generally uses static random access memory (SRAM).
  • SRAM static random access memory
  • the present disclosure provides a processing device for performing convolutional neural network operations, which can reduce the power consumption and circuit area of the processing device for performing convolutional neural network operations.
  • the embodiment of the present invention provides a processing device for performing convolutional neural network operations.
  • This convolutional neural network operation includes multiple convolutional layers.
  • the processing device includes an internal memory and a calculation circuit.
  • the calculation circuit performs the convolution operation of each convolution layer.
  • the internal memory is coupled to the calculation circuit and includes a plurality of memory cells, and is used to store the weight data of the convolutional layer.
  • Each memory cell includes a control circuit and a capacitor.
  • the control circuit has a leakage current path. The data retention time of each memory cell is determined according to the leakage current on the leakage current path and the capacitance value of the capacitor, and the data retention time is greater than the preset required time.
  • the embodiment of the present invention provides a processing method for performing convolutional neural network operations, which is suitable for processing devices including internal memory.
  • the method includes the following steps.
  • the weight data of at least one convolutional layer is obtained from the external memory through the internal memory, and the convolution operation of the convolutional layer is performed.
  • the internal memory includes multiple memory cells.
  • Each memory cell includes a control circuit and a capacitor.
  • the control circuit has a leakage current path.
  • the data retention time of each memory cell is determined according to the leakage current on the leakage current path and the capacitance value of the capacitor, and the data retention time is greater than the preset required time.
  • the data retention time of the memory cell of the internal memory is determined according to the leakage current on the leakage current path and the capacitance value of the capacitor, and the data retention time is greater than the preset required time.
  • these weight data will only become invalid when the internal memory is retained for a period of time.
  • the overall chip power consumption of the processing device including the internal memory can be reduced and the circuit area can be reduced.
  • FIG. 1 is a schematic diagram of a computing system for performing convolutional neural network operations according to an embodiment of the present invention
  • Fig. 2 is a schematic diagram of a convolutional neural network model according to an embodiment of the present invention.
  • Fig. 3 is a schematic diagram of a convolution operation according to an embodiment of the present invention.
  • Fig. 4 is a schematic diagram of a processing device according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of an internal storage device according to an embodiment of the present invention.
  • Fig. 6A is a schematic diagram of a memory cell according to an embodiment of the present invention.
  • 6B is a schematic diagram of a memory cell according to an embodiment of the invention.
  • FIG. 7 is a schematic diagram of data retention time according to an embodiment of the present invention.
  • FIG. 8 is a flowchart of a processing method for performing convolutional neural network operations according to an embodiment of the present invention.
  • processing device 110: processing device
  • WM, WM_1 ⁇ WM_5 convolution kernel
  • MC memory cell
  • L1, L2 leakage current path
  • Amp1 sense amplifier circuit
  • Amp2 Write amplifier circuit
  • ⁇ T data retention time
  • FIG. 1 is a schematic diagram of a computing system for performing convolutional neural network operations according to an embodiment of the present invention.
  • the computing system 10 can analyze input data based on a convolutional neural network operation to extract effective information.
  • the computing system 10 can be installed in various electronic terminal devices to implement various application functions.
  • the computing system 10 can be installed in a smart phone, a tablet computer, a medical device, or a robot device, and the present invention is not limited thereto.
  • the computing system 10 may analyze the fingerprint image or palmprint image sensed by the fingerprint sensing device based on a convolutional neural network operation to obtain information related to the sensed fingerprint and palmprint.
  • the computing system 10 may include a processing device 110 and an external memory 120.
  • the processing device 110 and the external memory 120 can communicate via the bus 130.
  • the processing device 110 may be implemented as a system chip.
  • the processing device 110 may perform a convolutional neural network operation according to the received input data, where the convolutional neural network operation includes a plurality of convolutional layers.
  • the present invention does not limit the neural network model corresponding to the convolutional neural network operation. It can be any neural network model including multiple convolutional layers, such as GoogleNet model, AlexNet model, VGGNet model, Various convolutional neural network models such as ResNet model and LeNet model.
  • the external memory 120 is coupled to the processing device 110 for recording various parameters required by the processing device 110 to perform convolutional neural network operations, such as the weight data of each convolutional layer and so on.
  • the external memory 120 may include dynamic random access memory (DRAM), flash memory (flash memory), or other memories.
  • the processing device 110 can read various parameters required for performing convolutional neural network operations from the external memory 120 to perform convolutional neural network operations on input data.
  • Fig. 2 is a schematic diagram of a convolutional neural network model according to an embodiment of the present invention.
  • the processing device 110 can input the input data d_i to the convolutional neural network model 20 to generate output data d_o.
  • the input data d_i may be a grayscale image or a color image.
  • the input data d_i can be a fingerprint sensing image or a palmprint sensing image.
  • the output data d_o can be a classification category that classifies the input data d_i, a segmented image that has undergone semantic segmentation, or image data that has undergone image processing (such as style conversion, image filling, or resolution optimization, etc.). This is not limited.
  • the convolutional neural network model 20 may include multiple layers, and these layers may include multiple convolutional layers. In some embodiments, these layers may also include a pooling layer, an excitation layer, a fully connected layer, etc., which are not limited in the present invention.
  • Each layer in the convolutional neural network model 20 can receive input data d_i or a feature map (feature map) generated by the previous layer to perform relative arithmetic processing to generate an output feature map or output data d_o.
  • the feature map is data used to express various features of the input data d_i, which can be in the form of a two-dimensional matrix or a three-dimensional matrix (also called a tensor).
  • FIG. 2 only shows that the convolutional neural network model 20 includes the convolutional layers L1 to L3 as an example for description.
  • the feature maps FM1, FM2, FM3 generated by the convolutional layers L1 to L3 are in the form of a three-dimensional matrix.
  • the feature maps FM1, FM2, FM3 may have a width w (or called a column), a height h (or called a row), and a depth d (or called a channel number).
  • the convolution layer L1 can generate a feature map FM1 by performing a convolution operation on the input data d_i according to one or more convolution kernels.
  • the convolution layer L2 may perform a convolution operation on the feature map FM1 according to one or more convolution kernels to generate the feature map FM2.
  • the convolution layer L3 may perform a convolution operation on the feature map FM2 according to one or more convolution kernels to generate the feature map FM3.
  • the convolution kernels used in the above convolution layers L1 to L3 may also be referred to as weight data, which may be in the form of a two-dimensional matrix or a three-dimensional matrix.
  • the convolutional layer L2 can perform a convolution operation on the feature map FM1 according to the convolution kernel WM.
  • the number of channels of the convolution kernel WM is the same as the depth of the feature map FM1.
  • the convolution kernel WM slides in the feature map FM1 according to a fixed step size. Whenever the convolution kernel WM is shifted, each weight included in the convolution kernel WM will be multiplied by all the feature values of the overlapping region on the feature map FM1 and then added. Since the convolution layer L2 performs a convolution operation on the feature map FM1 according to the convolution kernel WM, the feature value corresponding to a channel in the feature map FM2 can be generated.
  • FIG. 2 only takes a single convolution kernel WM as an example for illustration, but the convolution layer L2 can actually perform convolution operations on the feature map FM1 based on multiple convolution kernels to generate a feature map FM2 with multiple channels.
  • FIG. 3 is a schematic diagram of a convolution operation according to an embodiment of the invention.
  • a certain convolutional layer performs convolution operation on the feature map FM_i generated by the previous layer, and suppose that the convolutional layer has 5 convolution kernels WM_1 to WM_5.
  • These convolution kernels WM_1 to WM_5 are the weight data of the convolution layer.
  • the feature map FM_i has a height H1, a width W1, and M channels.
  • the convolution kernels WM_1 to WM_5 have height H2, width W2, and M channels.
  • the convolution layer uses the convolution kernel WM_1 and the feature map FM_i to perform a convolution operation to obtain the sub-feature map 31 belonging to the first channel in the feature map FM_(i+1).
  • the convolution layer uses the convolution kernel WM_2 and the feature map FM_i to perform a convolution operation to obtain the sub-feature map 32 belonging to the second channel in the feature map FM_(i+1). So on and so forth.
  • the sub-feature maps 31-35 corresponding to the convolution kernels WM_1 ⁇ WM_5 can be generated, thereby generating a feature map FM_ with height H3, width W3 and 5 channels. (i+1).
  • the processing device 110 for performing convolutional neural network operations needs to perform convolution operations based on the weight data.
  • the weight data may be stored in the external memory 120 or other storage devices in advance.
  • the external memory 120 may provide these weight data to the processing device 110. That is, the internal memory built in the processing device 110 can be used to store the weight data provided by the external memory 120.
  • Fig. 4 is a schematic diagram of a processing device according to an embodiment of the present invention.
  • the processing device 110 may include an internal memory 111, a calculation circuit 112, and a controller 113.
  • the internal memory 111 is also called the on-chip memory.
  • the internal memory 111 is coupled to the calculation circuit 112.
  • the storage capacity of the internal memory 111 is smaller than the storage capacity of the external memory 120.
  • the calculation circuit 112 is used to perform layer operations of multiple layers in the convolutional neural network operation, and it may include arithmetic logic circuits for completing various layer operations. It can be understood that the calculation circuit 112 may include an arithmetic logic circuit such as a multiplier array, an accumulator array, etc., to complete a convolution operation. In addition, the calculation circuit 112 may include a weight buffer 41. The weight buffer is used to temporarily store the weight data provided by the internal memory 111, so that the arithmetic logic circuit in the calculation circuit 112 can efficiently perform convolution operations.
  • the controller 113 can be implemented by a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), or other calculations.
  • the circuit is implemented, which can control the overall operation of the processing device 110.
  • the controller 113 can manage the operation parameters required for the operation of the convolutional neural network, such as weight data, so that the processing device 110 can normally perform the operation of each layer in the operation of the convolutional neural network.
  • the controller 113 may control the internal memory 111 to obtain the weight data of all convolutional layers from the external memory 120. In some embodiments, the controller 113 may control the internal memory 111 to obtain the weight data of different convolutional layers from the external memory 120 at different time points. For example, the controller 113 may control the internal memory 111 to obtain the weight data of the first convolutional layer from the external memory 120 at a first time point, and control the internal memory 111 to obtain the second convolutional layer from the external memory 120 at a second time point. The weight data of the layer, where the first time point is different from the second time point. At the second time point, the weight data of the first convolutional layer in the internal memory 111 will be updated to the weight data of the second convolutional layer.
  • the controller 113 may control the internal memory 111 to obtain different parts of the weight data of the same convolutional layer from the external memory 120 at different time points. For example, the controller 113 may control the internal memory 111 to obtain the first part of the weight data of the first convolutional layer from the external memory 120 at the first time point, and control the internal memory 11 to obtain the first part of the weight data of the first convolutional layer from the external memory 120 at the second time point. Obtain the second part of the weight data of the same first convolutional layer, where the first time point is different from the second time point.
  • all weight data required for convolutional neural network operations can be written into the internal memory 111 together, and the weight data required for convolutional neural network operations can be divided into multiple parts and written into the internal memory 111 at different time points.
  • the data in the internal memory 111 for storing the weight data of the convolutional neural network operation and the intermediate operation result (for example, the feature map of each convolutional layer) is frequently updated.
  • the data recorded by the memory cell of the internal memory 111 can be allowed to elapse with time. That is, the weight data calculated by the convolutional neural network can be kept in the internal memory 111 for a period of time.
  • each memory cell has a corresponding data retention time. After data is written into a certain memory cell of the internal memory 111, the written data can be retained in the memory cell until the data retention time expires. That is, the weight data recorded by the memory cell will become invalid when the data retention time expires. Examples will be listed below for clear description.
  • FIG. 5 is a schematic diagram of an internal storage device according to an embodiment of the invention.
  • the internal memory 111 may include a memory cell array 51, a row decoder 52, and a column decoder 53.
  • the memory cell array 51 a plurality of word lines WL and bit lines BL are mainly arranged alternately in an array manner, and each interlace point has a memory cell (Memory Cell) MC. That is, the memory cell array 51 includes a plurality of memory cells MC arranged in an array. These memory cells MC use the charge and discharge principle of capacitors to achieve the purpose of recording data.
  • the internal memory 111 receives the Access Row Address, it will be decoded by the row decoder 52 to enable the corresponding word line WL.
  • the memory cell MC in the memory cell array 51 can be used to store the weight data of one or more convolutional layers. That is, the weight data of one or more convolutional layers can be written into multiple memory cells MC in the memory cell array 51, and the weight data of one or more convolutional layers can be read from multiple memories in the memory cell array 51. The cell MC is read.
  • FIG. 6A is a schematic diagram of a memory cell according to an embodiment of the invention.
  • each memory cell MC in the memory cell array 51 may include a control circuit 61 and a capacitor C1.
  • the control circuit 61 may include a transistor M1. The control terminal of the transistor M1 is coupled to the word line WL of the internal memory 111, the first terminal of the transistor M1 is coupled to the bit line BL of the internal memory 111, and the second terminal of the transistor M1 is coupled to one terminal of the capacitor C1.
  • the control circuit 61 may also include other electronic components, which is not limited by the present invention.
  • the internal memory 111 uses the amount of charge stored in the capacitor C1 to represent a binary bit '1' or '0'.
  • the control circuit 61 may have a leakage current path, and the charge in the capacitor C1 may leak from the leakage current path of the control circuit 61.
  • the data retention time of each memory cell MC is determined according to the leakage current on the leakage current path and the capacitance value of the capacitor C1, wherein the data retention time is greater than a predetermined required time. The preset required time is determined according to the calculation speed and calculation amount of the calculation circuit 112.
  • FIG. 6B is a schematic diagram of a memory cell according to an embodiment of the invention.
  • each memory cell MC in the memory cell array 51 may include a capacitor C1, a switch SW1, a switch SW2, a sense amplifier circuit Amp1, and a write amplifier circuit Amp2.
  • One end of the switch SW1 is coupled to one end of the capacitor C1, and the other end of the switch SW1 can be coupled to the bit line BL of the internal memory 111.
  • One end of the switch SW2 is coupled to one end of the capacitor C1, and the other end of the switch SW2 is coupled to the input end of the sense amplifier circuit Amp1.
  • the other end of the capacitor C1 can be coupled to the reference ground voltage.
  • the output terminal of the sense amplifier circuit Amp1 can be coupled to the bit line BL of the internal memory 111.
  • the output terminal of the write amplifier circuit Amp2 is coupled to one end of the switch SW2 and the input terminal of the sense amplifier circuit Amp1, and the input terminal of the write amplifier circuit Amp2 is coupled to the bit line BL of the internal memory 111.
  • the control ends of the switch SW1 and the switch SW2 can be coupled to the word line WL of the internal memory 111.
  • the internal memory 111 uses the amount of charge stored in the capacitor C1 to represent a binary bit "1" or "0".
  • the switch SW1 or the switch SW2 can be turned on, so that the written data can be recorded in the capacitor C1 via the switch SW1 or the write amplifier circuit Amp2 and the switch SW2.
  • the switch SW2 can be turned on, so that the data recorded by the capacitor C1 can be read via the sense amplifier circuit Amp1.
  • the capacitor C1 will have a leakage phenomenon and generate a leakage current path L1 (indicated by the leakage current source 65 here), so that the data recorded by the capacitor C1 is lost.
  • the switch SW2 will leak and generate a leakage current path L2 (represented by the leakage current source 66 here), causing the data recorded by the capacitor C1 to be lost.
  • the leakage current levels of the leakage current source 65 and the leakage current source 66 depend on the component characteristics of the capacitor C1 and the switch SW2.
  • the calculation circuit 112 after the calculation circuit 112 obtains the weight data of one or more convolutional layers from the internal memory 111, the weight data recorded by each memory cell MC becomes invalid when the data retention time expires.
  • the weight data of the convolutional layer may include part or all of the weight values in at least one convolution kernel.
  • the calculation circuit 112 After the weight data is written into the memory cell MC, during the data retention time of the memory cell MC, the calculation circuit 112 will obtain the correct weight data from the memory cell MC, and temporarily store the weight data in the weight buffer 41 for subsequent volumes. Product operation is used. Moreover, after the data retention time of the memory cell MC has elapsed, too much charge leakage of the capacitor C1 in the memory cell MC causes the weight data recorded by the memory cell MC to be invalid.
  • the data retention time of each memory cell MC is positively related to the capacitance value of the capacitor C1. That is, the smaller the capacitance value of the capacitor C1, the shorter the data retention time of the memory cell MC. Conversely, the larger the capacitance value of the capacitor C1, the longer the data retention time of the memory cell MC. Based on this, in the case of ensuring that the data retention time is greater than the preset required time, even the use of a capacitor C1 with a small capacitance value is allowable, and thus the power consumption and circuit area of the memory reading can be reduced.
  • the data retention time of each memory cell MC is negatively related to the current value of the leakage current. That is, the smaller the capacitance value of the leakage current on the leakage current path provided by the control circuit 61 is, the longer the data retention time of the memory cell MC is. Conversely, the greater the capacitance value of the leakage current on the leakage current path provided by the control circuit 61, the shorter the data retention time of the memory cell MC. Based on this, under the condition that the data retention time is greater than the preset required time, the circuit configuration and internal component design of the control circuit 61 with leakage current path can be more flexible.
  • the internal memory 111 does not need to enter a refresh mode to perform a data refresh operation on each memory cell MC. Therefore, the circuit area of the internal memory 111 can also be reduced in the absence of related circuits required for the refresh mode.
  • the internal memory 111 obtains the weight data of one or more convolutional layers from the external memory 120.
  • the update speed of the weight data in the internal memory 111 should be increased. Therefore, in some embodiments, the weight data required for the convolutional neural network operation can be sequentially written into the internal memory 111 of the processing device 110 in batches to speed up the update speed of the weight data. In this case, the data amount of the weight data of the convolutional layer obtained from the external memory 120 is positively correlated with the capacitance value of the capacitor C1.
  • the internal memory 111 may first read the weight data of one of the multi-layer convolutional layers.
  • the internal memory 111 can retain the weight data of one of the multi-layer convolutional layers until the data retention time expires, and the weight data recorded in the internal memory 111 will become invalid when the data retention time expires.
  • the internal memory 111 reads the weight data of another layer of the multi-layer convolutional layer.
  • the internal memory 111 can retain the weight data of another layer of the multi-layer convolutional layer until the data retention time expires.
  • FIG. 7 is a schematic diagram of data retention time according to an embodiment of the invention.
  • the weight data of the convolutional layer is written into the internal memory 111.
  • the weight value in one or more convolution kernels of one of the multiple convolution layers can be written to the internal memory 111 at time t1.
  • part of the weight value in a convolution kernel of one of the multiple convolution layers can be written to the internal memory 111 at time t1.
  • the calculation circuit 112 reads the weight data of the convolutional layer from the internal memory 111.
  • the calculation circuit 112 After the calculation circuit 112 obtains the weight data of the convolutional layer from the internal memory 111, at time t3, the weight data recorded by each memory cell MC becomes invalid when the data retention time ⁇ T expires. After the weight data recorded by the memory cell MC becomes invalid, at time t4, other weight data of the convolutional layer is written into the memory cell MC of the internal memory 111. At time t5, the calculation circuit 112 reads other weight data of the convolutional layer from the internal memory 111. At time t6, other weight data recorded by each memory cell MC becomes invalid when the data retention time ⁇ T expires.
  • FIG. 8 is a flowchart of a processing method for performing convolutional neural network operations according to an embodiment of the present invention. Please refer to FIG. 8, the method of this embodiment is applicable to the processing device 110 in the embodiment of FIG.
  • step S801 the weight data of at least one convolutional layer is obtained from the external memory 120 through the internal memory 111, and the convolution operation of the convolutional layer is performed.
  • the processing device 110 may obtain the weight data of the first convolutional layer from the external memory 120 at the first time point through the internal memory 111, and obtain the weight data of the first convolutional layer from the external memory 120 at the second time point through the internal memory 111 The weight data of the second convolutional layer, where the first time point is different from the second time point.
  • the processing device 110 may obtain the first part of the weight data of the first convolutional layer from the external memory 120 through the internal memory 111 at a first time point, and obtain the first part of the weight data of the first convolutional layer from the external memory 120 through the internal memory 111 Obtain the second part of the weight data of the first convolutional layer, where the first time point is different from the second time point.
  • each memory cell in the internal memory 111 includes a control circuit and a capacitor.
  • the control circuit has a leakage current path, and the data retention time of each memory cell is determined according to the leakage current on the leakage current path and the capacitance value of the capacitor.
  • the memory cell of the internal memory used to record the weight data of the convolutional layer has a data retention time. After the data retention time has elapsed, the weight data recorded by the memory cell will become invalid due to the leakage of the capacitor.
  • the data retention time of the memory cell is determined by the leakage current and the capacitance value of the capacitor. Based on this, in the case of ensuring that the data retention time of the memory cell is greater than the preset required time, the memory cell can use a capacitor with a smaller capacitance value, thereby reducing the power consumption and circuit area of memory reading. Therefore, the circuit area and power consumption of the internal memory provided in the processing device can be reduced.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Semiconductor Memories (AREA)
  • Error Detection And Correction (AREA)
  • Image Analysis (AREA)

Abstract

一种用于执行卷积神经网络运算的处理装置与处理方法。卷积神经网络运算包括多个卷积层。处理装置包括内部存储器与计算电路。计算电路执行各卷积层的卷积运算。内部存储器耦接计算电路并包括多个记忆胞，并用以存储卷积层的权重数据。各记忆胞包括控制电路与电容器，控制电路具有漏电流路径，各记忆胞的数据保留时间依据漏电流路径上的漏电流与电容器的电容值而决定。

Description

用于执行卷积神经网络运算的处理装置与处理方法 技术领域
本揭露涉及一种计算装置，且特别是有关于一种用于执行卷积神经网络运算的处理装置与处理方法。
背景技术
人工智能近年得到迅速发展，极大地影响了人们的生活。基于人工神经网络，尤其是卷积神经网络(Convolutional Neural Network，CNN)在很多应用中的发展日趋成熟，例如在计算机视觉领域中得到广泛使用。随着卷积神经网络的应用越来越广泛，越来越多的芯片设计厂商开始设计用于执行卷积神经网络运算的处理芯片。执行卷积神经网络运算的处理芯片需要复杂的运算与庞大的参数量来分析输入数据。对于用于执行卷积神经网络运算的处理芯片而言，为了加速处理速度与降低重复存取外部存储器所产生的功耗，处理芯片内部一般设置有内部存储器(又称为芯片内建存储器(on-chip-memory))来存储暂时计算结果与卷积运算所需的权重数据。一般而言，此内部存储器普遍使用静态随机存取存储器(static random access memory，SRAM)。然而，当静态随机存取存储器内的数据基于卷积神经网络运算的特性而被频繁读写时，将导致处理芯片的整体芯片功耗上升。
揭露内容
有鉴于此，本揭露提供一种用于执行卷积神经网络运算的处理装置，其可降低用于执行卷积神经网络运算的处理装置的功耗与其电路面积。
本发明实施例提出一种用于执行卷积神经网络运算的处理装置。此卷积神经网络运算包括多个卷积层。处理装置包括内部存储器与计算电路。计算电路执行各卷积层的卷积运算。内部存储器耦接计算电路并包括多个记忆胞，并用以存储卷积层的权重数据。各记忆胞包括控制电路与电容器，控制电路具有漏电流路径，各记忆胞的数据保留时间依据漏电流路径上的漏电流与电容器的电容值而决定，且数据保留时间大于预设需求时间。
本发明实施例提出一种用于执行卷积神经网络运算的处理方法，适用于包括内部存储器的处理装置。所述方法包括下列步骤。透过内部存储器自外部存储器获取至少一卷积层的权重数据，并执行卷积层的卷积运算。其中，内部存储器包括多个记忆胞。各记忆胞包括控制电路与电容器，控制电路具有漏电流路径，各记忆胞的数据保留时间依据漏电流路径上的漏电流与电容器的电容值而决定，且数据保留时间大于预设需求时间。
基于上述，于本发明的实施例中，内部存储器的记忆胞的数据保留时间是依据漏电流路径上的漏电流与电容器的电容值而决定，且此数据保留时间会大于预设需求时间。换言之，在确保内部存储器中的权重数据被计算电路获取的条件下，这些权重数据只会在内部存储器保留一段时间就失效。基此，在让内部存储器所记录的权重数据可随时间失效的情况下，此包括内部存储器的处理装置的整体芯片功耗可以下降并减少电路面积。
为让本揭露的上述特征和优点能更明显易懂，下文特举实施例，并配合所附图式作详细说明如下。
附图说明
图1是依照本发明一实施例的执行卷积神经网络运算的计算系统的示意图；
图2是依照本发明一实施例的卷积神经网络模型的示意图；
图3是依照本发明一实施例的卷积运算的示意图；
图4是依照本发明一实施例的处理装置的示意图；
图5是依照本发明一实施例的内部存储装置的示意图；
图6A是依照本发明一实施例的记忆胞的示意图；
图6B是依照本发明一实施例的记忆胞的示意图；
图7是依照本发明一实施例的数据保留时间的示意图；
图8是依照本发明一实施例的用于执行卷积神经网络运算的处理方法的流程图。
附图标记说明
10:计算系统；
110:处理装置；
120:外部存储器；
130:总线；
d_i:输入数据；
d_o:输出数据；
20:卷积神经网络模型；
L1～L3:卷积层；
FM1、FM2、FM3、FM_i、FM_(i+1):特征图；
WM、WM_1～WM_5:卷积核；
31～35:子特征图；
111:内部存储器；
112:计算电路；
113:控制器；
41:权重缓冲器；
51:记忆胞阵列；
52:行解码器；
53:列解码器；
WL:字线；
BL:位线；
MC:记忆胞；
61:控制电路；
M1:晶体管；
C1:电容器；
SW1,SW2:开关；
L1,L2:漏电流路径；
65,66:漏电流源；
Amp1:读出放大器电路；
Amp2:写入放大器电路；
ΔT:数据保留时间；
t1～t6:时间；
S801:流程步骤。
具体实施方式
现将详细地参考本揭露的示范性实施例，示范性实施例的实例说明于附图中。只要有可能，相同组件符号在图式和描述中用来表示相同或相似部分。
应当理解，当诸如层、膜、区域或基板的组件被称为在另一组件“上”或“连接到”另一组件时，其可以直接在另一组件上或与另一组件连接，或者中间组件可以也存在。相反，当组件被称为“直接在另一组件上”或“直接连接到”另一组件时，不存在中间组件。如本文所使用的，“连接”可以指物理及/或电性连接。再者，“电性连接”或“耦合”可以是二组件间存在其它组件。
图1是依照本发明一实施例的执行卷积神经网络运算的计算系统的示意图。请参照图1，计算系统10可基于卷积神经网络运算来分析输入数据以提取有效信息。计算系统10可安装于各式电子终端设备中以实现各种不同的应用功能。举例而言，计算系统10可安装于智能型手机、平板计算机、医疗设备或机器人设备中，本发明对此不限制。于一实施例中，计算系统10可基于卷积神经网络运算来分析指纹感测装置所感测的指纹图像或掌纹图像，以获取与感测指纹与掌纹相关的信息。
计算系统10可包括处理装置110以及外部存储器120。处理装置110以及外部存储器120可经由总线130进行通讯。于一实施例中，处理装置110可被实施为一系统芯片。处理装置110可依据接收到的输入数据执行卷积神经网络运算，其中卷积神经网络运算包括多个卷积层。需说明的是，本发明对于卷积神经网络运算所对应的神经网络模型并不加以限制，其可以为任何包括多个卷积层的神经网络模型，像是GoogleNet模型、AlexNet模型、VGGNet模型、ResNet模型、LeNet模型等各种卷积神经网络模型。
外部存储器120耦接处理装置110，用于记录处理装置110执行卷积神经网络运算所需的各种参数，像是各个卷积层的权重数据等等。外部存储器120可以包含动态随机存取存储器(dynamic random access memory，DRAM)、闪存(flash memory)或是其他存储器。处理装置110可从外部存储器120读取执行卷积神经网络运算所需的各种参数，以对输入数据执行卷积神经网络运算。
图2是依照本发明一实施例的卷积神经网络模型的示意图。请参照图2， 处理装置110可将输入数据d_i输入至基于卷积神经网络模型20而产生输出数据d_o。于一实施例中，输入数据d_i可以是一张灰阶图像或彩色图像。从另一方面来看，输入数据d_i可以是一张指纹感测图像或掌纹感测图像。输出数据d_o可以是对输入数据d_i进行分类的分类类别、经过语义分割的分割图像，或是经过图像处理(例如风格转换、图像填补或分辨率优化等等)的图像数据等等，本发明对此不限制。
卷积神经网络模型20可包括多个层，而这些层可包括多个卷积层。于一些实施例中，这些层还可包括池化层、激励层与全连接层等等，本发明对此不限制。卷积神经网络模型20中的每一层可接收输入数据d_i或前层产生的特征图(feature map)，以执行相对的运算处理以产生输出特征图或输出数据d_o。于此，特征图为用以表达输入数据d_i的各种特征的数据，其可为二维矩阵形式或三维矩阵(亦可称为张量(tensor))形式。
为了方便说明，图2仅绘示了卷积神经网络模型20包括卷积层L1～L3为范例进行说明。如图2所示，卷积层L1～L3所产生的特征图FM1、FM2、FM3为三维矩阵形式。于本范例中，特征图FM1、FM2、FM3可具有宽度w(或称为列)、高度h(或称为行)，以及深度d(或称为通道数量)。
卷积层L1可依据一或多个卷积核对输入数据d_i进行卷积运算而产生特征图FM1。卷积层L2可依据一或多个卷积核对特征图FM1进行卷积运算而产生特征图FM2。卷积层L3可依据一或多个卷积核对特征图FM2进行卷积运算而产生特征图FM3。上述卷积层L1～L3所使用的卷积核又可称为权重数据，其可为二维矩阵形式或三维矩阵形式。举例而言，卷积层L2可依据卷积核WM对特征图FM1进行卷积运算。于一些实施例中，卷积核WM的通道数目与特征图FM1的深度相同。卷积核WM在特征图FM1依据固定步长进行滑动。每当卷积核WM移位，卷积核WM中所包含的每一权重将与特征图FM1上重合的区的所有特征值相乘后相加。由于卷积层L2依据卷积核WM对特征图FM1进行卷积运算，因此可产生特征图FM2中对应至一个通道的特征值。图2仅以单一个卷积核WM为示范例进行说明，但卷积层L2实际上可依据多个卷积核对特征图FM1进行卷积运算，以产生具有多个通道的特征图FM2。
图3是依照本发明一实施例的卷积运算的示意图。请参照图3，假设某 一层卷积层对前层所产生的特征图FM_i进行卷积运算，且假设该层卷积层具有5个卷积核WM_1～WM_5。这些卷积核WM_1～WM_5为该卷积层的权重数据。特征图FM_i具有高度H1、宽度W1以及M个通道。卷积核WM_1～WM_5具有高度H2、宽度W2以及M个通道。该卷积层使用卷积核WM_1与特征图FM_i进行卷积运算，可获取特征图FM_(i+1)中属于第一个通道的子特征图31。该卷积层使用卷积核WM_2与特征图FM_i进行卷积运算，可获取特征图FM_(i+1)中属于第二个通道的子特征图32。依此类推。基于此卷积层具有5个卷积核WM_1～WM_5，因而可产生卷积核WM_1～WM_5分别对应的子特征图31～35，从而产生具有高度H3、宽度W3以及5个通道的特征图FM_(i+1)。
基于图2与图3的说明可知，用以执行卷积神经网络运算的处理装置110需要依据权重数据进行卷积运算。于一些实施例中，这些权重数据可预先存储于外部存储器120或其他存储装置。外部存储器120可将这些权重数据提供给处理装置110。亦即，内建于处理装置110的内部存储器可用以存储外部存储器120所提供的权重数据。
图4是依照本发明一实施例的处理装置的示意图。请参照图4，处理装置110可包括内部存储器111、计算电路112，以及控制器113。内部存储器111又称为芯片内建存储器。内部存储器111耦接计算电路112。于一些实施例中，内部存储器111的存储容量小于外部存储器120的存储容量。
计算电路112用以执行卷积神经网络运算中多个层的层运算，其可包括用以完成各种层运算的算术逻辑电路。可知的，计算电路112可包括乘法器阵列、累加器阵列等等用以完成卷积运算的算术逻辑电路。此外，计算电路112可包括权重缓冲器41。权重缓冲器用以暂存内部存储器111所提供的权重数据，以利计算电路112内的算术逻辑电路可有效率地进行卷积运算。
控制器113可以藉由中央处理器(Central Processing Unit，CPU)、微处理器、特殊应用集成电路(Application-specific integrated circuit,ASIC)、数字信号处理器(digital signal processor,DSP)或是其他计算电路来实施，其可控制处理装置110的整体运作。控制器113可管理卷积神经网络运算所需的运算参数，例如权重数据，以使处理装置110可正常地执行卷积神经网络运算中各个层的运算。
于一些实施例中，控制器113可控制内部存储器111从外部存储器120获取所有卷积层的权重数据。于一些实施例中，控制器113可控制内部存储器111从外部存储器120于不同时间点获取不同卷积层的权重数据。举例而言，控制器113可控制内部存储器111从外部存储器120于第一时间点获取第一卷积层的权重数据，并控制内部存储器111从外部存储器120于第二时间点获取第二卷积层的权重数据，其中第一时间点相异于第二时间点。于第二时间点，内部存储器111中第一卷积层的权重数据将被更新为第二卷积层的权重数据。于一些实施例中，控制器113可控制内部存储器111从外部存储器120于不同时间点获取同一卷积层的权重数据的不同部份。举例而言，控制器113可控制内部存储器111从外部存储器120于第一时间点获取第一卷积层的权重数据的第一部份，并控制内部存储器11从外部存储器120于第二时间点获取相同的第一卷积层的权重数据的第二部份，其中第一时间点相异于第二时间点。
基于前述可知，卷积神经网络运算所需的所有权重数据可一起写入内部存储器111，卷积神经网络运算所需的权重数据可分成多个部份而依据于不同时间点写入内部存储器111。由此可知，用于存储卷积神经网络运算的权重数据与中间运算结果(例如各卷积层的特征图)的内部存储器111内的数据会频繁地更新。基此，于本发明实施例中，在确保内部存储器111内的权重数据可以被计算电路112取得的情况下，可容许内部存储器111的记忆胞所记录的数据随时间而消逝。亦即，卷积神经网络运算的权重数据于内部存储器111内保留一段时间即可。
更具体而言，于本发明的实施例中，依据内部存储器111的记忆胞的电路配置与组件特性，各记忆胞具有对应的数据保留时间。在数据写入内部存储器111的某一记忆胞之后，写入数据可保留于该记忆胞内直至数据保留时间期满。亦即，记忆胞所记录的权重数据会在数据保留时间期满时失效。以下将列举实施例以清楚说明。
图5是依照本发明一实施例的内部存储装置的示意图。请参照图5，内部存储器111可包括记忆胞阵列51、行解码器52，以及列解码器53。记忆胞阵列51中主要是由多条字线WL与位线BL以阵列方式交错排列，而每个交错点则有一记忆胞(Memory Cell)MC。亦即，记忆胞阵列51包括阵列排列 的多个记忆胞MC。这些记忆胞MC是利用电容器的充放电原理来达到记录数据的目的。当内部存储器111收到存取行地址(Access Row Address)时，会先经过行解码器52译码以致能对应的字线WL。于是，连接被致能字线WL的记忆胞MC内的电容器的电荷可流至对应的位线BL。列解码器53可依据列地址(column Address)控制行选择器，以将列地址所对应的数据读出或写入。需说明的是，于一些实施例中，记忆胞阵列51中的记忆胞MC可用以存储一或多个卷积层的权重数据。亦即，一或多个卷积层的权重数据可写入记忆胞阵列51中的多个记忆胞MC，且一或多个卷积层的权重数据可从记忆胞阵列51中的多个记忆胞MC被读出。
图6A是依照本发明一实施例的记忆胞的示意图。请参照图6A，记忆胞阵列51中的各记忆胞MC可包括控制电路61与电容器C1。于一些实施例中，控制电路61可包括晶体管M1。晶体管M1的控制端耦接内部存储器111的字线WL，且晶体管M1的第一端耦接内部存储器111的位线BL，晶体管M1的第二端耦接电容器C1的一端。然而，于其他实施例中，控制电路61还可包括其他电子组件，本发明对此不限制。于一些实施例中，内部存储器111是利用电容器C1内存储电荷的多寡来代表一个二进制位的‘1’或‘0’。
值得注意的是，即使记忆胞MC内的晶体管M1为关闭的状态，电容器C1所存储之电荷也会随时间逐渐消逝，造成数据流失。亦即，电容器C1会有漏电现象，使得其所记录的数据流失。更详细而言，控制电路61可具有漏电流路径，电容器C1内的电荷可能从控制电路61的漏电流路径漏掉。于本发明的实施例中，各记忆胞MC的数据保留时间是依据漏电流路径上的漏电流与电容器C1的电容值而决定，其中数据保留时间会大于一预设需求时间。预设需求时间是依据计算电路112的计算速度与计算量而决定。计算电路112的计算速度越高，则预设需求时间越短。计算电路112的计算量越低，则预设需求时间越短。可知的，当预设需求时间越短，记忆胞MC的数据保留时间也可以越短。
图6B是依照本发明一实施例的记忆胞的示意图。请参照图6B，于一些实施例中，记忆胞阵列51中的各记忆胞MC可包括电容器C1、开关SW1、开关SW2、读出放大器电路Amp1以及写入放大器电路Amp2。开关SW1的一端耦接电容器C1的一端，而开关SW1的另一端可耦接内部存储器111的 位线BL。开关SW2的一端耦接电容器C1的一端，开关SW2的另一端耦接读出放大器电路Amp1的输入端。电容器C1的另一端可耦接至参考地电压。读出放大器电路Amp1的输出端可耦接内部存储器111的位线BL。写入放大器电路Amp2的输出端耦接开关SW2的一端与读出放大器电路Amp1的输入端，写入放大器电路Amp2的输入端耦接内部存储器111的位线BL。开关SW1与开关SW2的控制端可耦接内部存储器111的字线WL。内部存储器111是利用电容器C1内存储电荷的多寡来代表一个二进制位的‘1’或‘0’。当要将数据写入电容器C1时，开关SW1或开关SW2可导通，使写入数据可经由开关SW1或写入放大器电路Amp2与开关SW2而记录于电容器C1。当要读出电容器C1所记录的数据时，开关SW2可导通，使电容器C1所记录的数据可经由读出放大器电路Amp1被读取。
如图6B所示，电容器C1会有漏电现象而产生漏电流路径L1(于此以漏电流源65表示)，使得电容器C1所记录的数据流失。此外，即便SW2没有导通，开关SW2会有漏电现象而产生漏电流路径L2(于此以漏电流源66表示)，使得电容器C1所记录的数据流失。于此，漏电流源65与漏电流源66的漏电流准位取决于电容器C1与开关SW2的组件特性。
于一些实施例中，在计算电路112自内部存储器111获取一或多个卷积层的权重数据之后，各记忆胞MC所记录的权重数据在数据保留时间期满时失效。于此，卷积层的权重数据可包括至少一卷积核中部份或全部权重值。在将权重数据写入记忆胞MC之后，在记忆胞MC的数据保留时间期间，计算电路112会从记忆胞MC获取正确的权重数据，并将权重数据暂存于权重缓冲器41以供后续卷积运算使用。并且，在经过记忆胞MC的数据保留时间之后，记忆胞MC内电容器C1的电荷漏失过多导致其所记录的权重数据已经失效。
于一些实施例中，各记忆胞MC的数据保留时间正相关于电容器C1的电容值。亦即，电容器C1的电容值越小，则记忆胞MC的数据保留时间越短。反之，电容器C1的电容值越大，则记忆胞MC的数据保留时间越长。基此，在确保数据保留时间大于预设需求时间的情况下，即便使用具备小电容值的电容器C1也是可允许的，因而可降低存储器读取的功耗与电路面积。
于一些实施例中，各记忆胞MC的数据保留时间负相关于漏电流的电流 值。亦即，控制电路61所提供之漏电流路径上漏电流的电容值越小，则记忆胞MC的数据保留时间越长。反之，控制电路61所提供之漏电流路径上漏电流的电容值越大，则记忆胞MC的数据保留时间越短。基此，在确保数据保留时间大于预设需求时间的情况下，具备漏电流路径的控制电路61的电路配置与内部组件设计可以更为弹性。
值得一提的是，相较于传统的动态随机存取存储器，内部存储器111不需要进入刷新(refresh)模式来对各记忆胞MC进行数据刷新动作。因此，在不具备刷新模式所需之相关电路的情况下，内部存储器111的电路面积也可因而降低。
此外，基于前述可知，内部存储器111自外部存储器120获取一或多个卷积层的权重数据。若要减少电容器C1的电容值且因而缩减记忆胞MC的数据保留时间，代表内部存储器111内的权重数据的更新速度要加快。因此，于一些实施例中，卷积神经网络运算所需的权重数据可分批依序写入处理装置110的内部存储器111，以加快权重数据的更新速度。在此情况下，自外部存储器120获取卷积层的权重数据的数据量正相关于电容器C1的电容值。
举例而言，若要使用具备小电容值的电容器C1来降低读取功耗时，内部存储器111可先读取多层卷积层其中一层的权重数据。内部存储器111可保留多层卷积层其中一层的权重数据直至数据保留时间期满，且内部存储器111所记录的权重数据会于数据保留时间期满时失效。之后，内部存储器111再读取多层卷积层其中另一层的权重数据。相似的，内部存储器111可保留多层卷积层其中另一层的权重数据直至数据保留时间期满。
图7是依照本发明一实施例的数据保留时间的示意图。请参照图7，于时间t1，卷积层的权重数据写入内部存储器111。例如，多个卷积层其中之一层的一或多个卷积核中的权重值可于时间t1写入至内部存储器111。或者，多个卷积层其中之一层的一个卷积核中的部份权重值可于时间t1写入至内部存储器111。于时间t2，计算电路112自内部存储器111读取卷积层的权重数据。在计算电路112自内部存储器111获取卷积层的权重数据之后，于时间点t3，各记忆胞MC所记录的权重数据在数据保留时间ΔT期满时失效。在记忆胞MC所记录的权重数据失效之后，于时间点t4，卷积层的其他权重数据写入的内部存储器111的记忆胞MC。于时间t5，计算电路112自内部存储 器111读取卷积层的其他权重数据。于时间点t6，各记忆胞MC所记录的其他权重数据在数据保留时间ΔT期满时失效。
图8是依照本发明一实施例的用于执行卷积神经网络运算的处理方法的流程图。请参照图8，本实施例的方式适用于图4之实施例中的处理装置110，以下即搭配处理装置110中的各项组件说明本实施例的详细步骤。
于步骤S801，透过内部存储器111自外部存储器120获取至少一卷积层的权重数据，并执行卷积层的卷积运算。于一些实施例中，处理装置110可透过内部存储器111从外部存储器120于第一时间点获取第一卷积层的权重数据，并透过内部存储器111从外部存储器120于第二时间点获取第二卷积层的权重数据，其中第一时间点相异于第二时间点。于一些实施例中，处理装置110可透过内部存储器111从外部存储器120于第一时间点获取第一卷积层的权重数据的第一部份，并透过内部存储器111从外部存储器120于获取第一卷积层的权重数据的第二部份，其中第一时间点相异于第二时间点。
需注意的是，内部存储器111中各记忆胞所记录的至少一卷积层的权重数据，例如某一卷积层的所有权重数据或部份权重数据，会在数据保留时间期满时失效。内部存储器111中各记忆胞包括控制电路与电容器。此控制电路具有漏电流路径，各记忆胞的数据保留时间依据漏电流路径上的漏电流与电容器的电容值而决定。
综上所述，于本发明实施例中，用以记录卷积层的权重数据的内部存储器的记忆胞具有数据保留时间。在经过数据保留时间之后，记忆胞所记录的权重数据会因为电容器的漏电现象而失效。记忆胞的数据保留时间是依据漏电流与电容器的电容值而决定。基此，在确保记忆胞的数据保留时间大于预设需求时间的情况下，记忆胞可使用具备较小电容值的电容器，因而可降低内存读取的功耗与电路面积。于是，设置于处理装置内的内部存储器的电路面积与消耗功率可以减少。
最后应说明的是：以上各实施例仅用以说明本揭露的技术方案，而非对其限制；尽管参照前述各实施例对本揭露进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本揭露各实施例技术方案的范围。

Claims (14)

  1. 一种用于执行卷积神经网络运算的处理装置，所述卷积神经网络运算包括多个卷积层，其特征在于，所述处理装置包括：
    计算电路，执行各所述卷积层的卷积运算；以及
    内部存储器，耦接所述计算电路并包括多个记忆胞，并用以存储所述卷积层的权重数据，
    其中，各所述记忆胞包括控制电路与电容器，所述控制电路具有漏电流路径，各所述记忆胞的数据保留时间依据所述漏电流路径上的漏电流与电容器的电容值而决定。
  2. 根据权利要求1所述的用于执行卷积神经网络运算的处理装置，其特征在于，在所述计算电路自所述内部存储器获取所述卷积层的权重数据之后，各所述记忆胞所记录的权重数据在所述数据保留时间期满时失效。
  3. 根据权利要求2所述的用于执行卷积神经网络运算的处理装置，其特征在于，在所述记忆胞所记录的权重数据失效之后，所述卷积层的其他权重数据写入的所述内部存储器的所述记忆胞。
  4. 根据权利要求1所述的用于执行卷积神经网络运算的处理装置，其特征在于，所述数据保留时间正相关于所述电容器的电容值。
  5. 根据权利要求1所述的用于执行卷积神经网络运算的处理装置，其特征在于，所述数据保留时间负相关于所述漏电流的电流值。
  6. 根据权利要求1所述的用于执行卷积神经网络运算的处理装置，其特征在于，所述内部存储器自外部存储器获取所述卷积层的权重数据。
  7. 根据权利要求6所述的用于执行卷积神经网络运算的处理装置，其特征在于，自外部存储器获取所述卷积层的权重数据的数据量正相关于所述电容器的电容值。
  8. 根据权利要求1所述的用于执行卷积神经网络运算的处理装置，其特征在于，所述卷积层的权重数据包括至少一卷积核中部份或全部权重值。
  9. 根据权利要求1所述的用于执行卷积神经网络运算的处理装置，其特征在于，所述控制电路包括晶体管，所述晶体管的控制端耦接所述内部存储器的字线，且所述晶体管的第一端耦接所述内部存储器的位线，所述晶体管的第二端耦接所述电容器的一端。
  10. 根据权利要求1所述的用于执行卷积神经网络运算的处理装置，其特征在于，所述计算电路包括权重缓冲器，所述内部存储器将所述卷积层的权重数据提供给所述权重缓冲器。
  11. 根据权利要求1所述的用于执行卷积神经网络运算的处理装置，其特征在于，所述计算电路用于分析指纹感测装置所感测的指纹图像或掌纹图像。
  12. 一种用于执行卷积神经网络运算的处理方法，适用于包括内部存储器的处理装置，其特征在于，所述方法包括：
    透过所述内部存储器自外部存储器获取至少一卷积层的权重数据，并执行所述卷积层的卷积运算，
    其中所述内部存储器包括多个记忆胞，各所述记忆胞包括控制电路与电容器，所述控制电路具有漏电流路径，各所述记忆胞的数据保留时间依据所述漏电流路径上的漏电流与电容器的电容值而决定。
  13. 根据权利要求12所述的用于执行卷积神经网络运算的处理方法，其特征在于，透过所述内部存储器自所述外部存储器获取所述至少一卷积层的权重数据的步骤包括：
    透过所述内部存储器从所述外部存储器于第一时间点获取第一卷积层的权重数据，并透过所述内部存储器从所述外部存储器于第二时间点获取第二卷积层的权重数据，其中所述第一时间点相异于所述第二时间点。
  14. 根据权利要求12所述的用于执行卷积神经网络运算的处理方法，其特征在于，透过所述内部存储器自所述外部存储器获取所述至少一卷积层的权重数据的步骤包括：
    透过所述内部存储器从所述外部存储器于第一时间点获取第一卷积层的权重数据的第一部份，并透过所述内部存储器从所述外部存储器于第二时间点获取所述第一卷积层的权重数据的第二部份，其中所述第一时间点相异于所述第二时间点。
PCT/CN2021/082078 2020-05-04 2021-03-22 用于执行卷积神经网络运算的处理装置与处理方法 WO2021223528A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202063019965P 2020-05-04 2020-05-04
US63/019,965 2020-05-04

Publications (1)

Publication Number Publication Date
WO2021223528A1 true WO2021223528A1 (zh) 2021-11-11

Family

ID=75930528

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/082078 WO2021223528A1 (zh) 2020-05-04 2021-03-22 用于执行卷积神经网络运算的处理装置与处理方法

Country Status (3)

Country Link
CN (1) CN112836815A (zh)
TW (2) TWM614073U (zh)
WO (1) WO2021223528A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113570034B (zh) * 2021-06-18 2022-09-27 北京百度网讯科技有限公司 处理装置、神经网络的处理方法及其装置
US20230068645A1 (en) * 2021-08-27 2023-03-02 Taiwan Semiconductor Manufacturing Company, Ltd. Memory device and method for computing-in-memory (cim)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107203808A (zh) * 2017-05-08 2017-09-26 中国科学院计算技术研究所 一种二值卷积装置及相应的二值卷积神经网络处理器
CN108764467A (zh) * 2018-04-04 2018-11-06 北京大学深圳研究生院 用于卷积神经网络卷积运算和全连接运算电路
US20190205735A1 (en) * 2017-12-29 2019-07-04 Facebook, Inc. Lowering hardware for neural networks
CN110880038A (zh) * 2019-11-29 2020-03-13 中国科学院自动化研究所 基于fpga的加速卷积计算的系统、卷积神经网络

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010025130A1 (en) * 2008-08-28 2010-03-04 Massachusetts Institute Of Technology Circuits and methods representative of spike timing dependent plasticity of neurons
CN106503796B (zh) * 2015-10-08 2019-02-12 上海兆芯集成电路有限公司 多运算神经网络单元
CN105760930B (zh) * 2016-02-18 2018-06-05 天津大学 用于aer的多层脉冲神经网络识别系统
US10515302B2 (en) * 2016-12-08 2019-12-24 Via Alliance Semiconductor Co., Ltd. Neural network unit with mixed data and weight size computation capability
US11594770B2 (en) * 2017-05-03 2023-02-28 Semiconductor Energy Laboratory Co., Ltd. Neural network, power storage system, vehicle, and electronic device
US10910061B2 (en) * 2018-03-14 2021-02-02 Silicon Storage Technology, Inc. Method and apparatus for programming analog neural memory in a deep learning artificial neural network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107203808A (zh) * 2017-05-08 2017-09-26 中国科学院计算技术研究所 一种二值卷积装置及相应的二值卷积神经网络处理器
US20190205735A1 (en) * 2017-12-29 2019-07-04 Facebook, Inc. Lowering hardware for neural networks
CN108764467A (zh) * 2018-04-04 2018-11-06 北京大学深圳研究生院 用于卷积神经网络卷积运算和全连接运算电路
CN110880038A (zh) * 2019-11-29 2020-03-13 中国科学院自动化研究所 基于fpga的加速卷积计算的系统、卷积神经网络

Also Published As

Publication number Publication date
TWM614073U (zh) 2021-07-01
TWI751931B (zh) 2022-01-01
CN112836815A (zh) 2021-05-25
TW202143116A (zh) 2021-11-16

Similar Documents

Publication Publication Date Title
US11501109B2 (en) Non-volatile memory die with on-chip data augmentation components for use with machine learning
US11687454B2 (en) Memory circuit and cache circuit configuration
CN112992217B (zh) 在多芯片封装中使用温度偏差来控制操作的存储器器件
TWI766568B (zh) 用於執行卷積神經網路運算的處理裝置與其操作方法
WO2021223528A1 (zh) 用于执行卷积神经网络运算的处理装置与处理方法
US11520521B2 (en) Storage controller having data augmentation components for use with non-volatile memory die
TWI759884B (zh) 記憶體裝置、記憶體裝置的操作方法以及記憶體系統
US9524772B2 (en) Memory device of a single-ended bitline structure including reference voltage generator
WO2020073801A1 (zh) 一种3d图像处理中数据读写方法及系统、存储介质及终端
CN116030858A (zh) 存储设备以及用于控制行锤击的方法
US9455036B1 (en) System architectures with data transfer paths between different memory types
US11567690B2 (en) Semiconductor memory device and electronic system the same
US11942137B2 (en) Memory controller and memory system including the same
CN111801736A (zh) 具有非均匀刷新的存储器设备
US7386654B2 (en) Non-volatile configuration data storage for a configurable memory
TWI833945B (zh) 記憶體裝置、高頻寬記憶體系統及提升高頻寬記憶體的整體效能的方法
US20230195327A1 (en) Memory system and method of operating the same
US20230395132A1 (en) Sram cell configured to perform multiply-accumulate (mac) operation on multi-bit data based on charge sharing and method of operating the same
US20240096391A1 (en) Memory devices and methods thereof for managing row hammer events therein
US20230236732A1 (en) Memory device
US20170287542A1 (en) Methods and apparatus for memory programming
JPS62264495A (ja) １サイクルで論理値を書込むことのできるダイナミツクメモリ
WO2020082451A1 (zh) 一种基于静态随机存取存储器快速搜寻有效闪存页的方法
KR20240115586A (ko) 리프레쉬 동작을 관리하는 메모리 컨트롤러 및 그 동작방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21800268

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21800268

Country of ref document: EP

Kind code of ref document: A1